Max level shown:
12024-12-14T04:45:01.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f792763c-5a9a-4c29-b193-21b24a8bdeca roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
22024-12-14T04:45:01.256ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f792763c-5a9a-4c29-b193-21b24a8bdeca resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32024-12-14T04:45:01.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3065 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f792763c-5a9a-4c29-b193-21b24a8bdeca response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42024-12-14T04:45:02.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ff162b11-5d1b-45d5-998d-8b6d365e640f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
52024-12-14T04:45:02.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ff162b11-5d1b-45d5-998d-8b6d365e640f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
62024-12-14T04:45:02.289ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2008 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ff162b11-5d1b-45d5-998d-8b6d365e640f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
72024-12-14T04:45:02.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1342 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = e5fd104d-7c0c-48c6-a810-890fe5dad944 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
82024-12-14T04:45:03.310ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b3ff876-d162-4e51-93d4-80810221f595 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
92024-12-14T04:45:03.310ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b3ff876-d162-4e51-93d4-80810221f595 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
102024-12-14T04:45:03.312ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2111 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b3ff876-d162-4e51-93d4-80810221f595 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
112024-12-14T04:45:04.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 90084f6c-78d1-4284-bd8b-22169bda9e16 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
122024-12-14T04:45:04.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 90084f6c-78d1-4284-bd8b-22169bda9e16 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
132024-12-14T04:45:04.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2600 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 90084f6c-78d1-4284-bd8b-22169bda9e16 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
142024-12-14T04:45:05.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11af10b9-999e-42bb-81fc-2c3eb62adc35 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
152024-12-14T04:45:05.356ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11af10b9-999e-42bb-81fc-2c3eb62adc35 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
162024-12-14T04:45:05.358ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2428 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11af10b9-999e-42bb-81fc-2c3eb62adc35 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
172024-12-14T04:45:06.378ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 081d3c1e-0dcf-4d52-9170-45d20545a947 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
182024-12-14T04:45:06.379ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 081d3c1e-0dcf-4d52-9170-45d20545a947 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
192024-12-14T04:45:06.381ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2358 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 081d3c1e-0dcf-4d52-9170-45d20545a947 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
202024-12-14T04:45:06.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = d78555fa-c54b-4bb9-ae82-6331c8d558f1 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
212024-12-14T04:45:06.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = d78555fa-c54b-4bb9-ae82-6331c8d558f1 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
222024-12-14T04:45:06.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5914 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = d78555fa-c54b-4bb9-ae82-6331c8d558f1 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
232024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
242024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
252024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
262024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
272024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
282024-12-14T04:45:06.882ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
292024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
302024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
312024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
322024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
332024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
342024-12-14T04:45:06.882ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
352024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
362024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
372024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
382024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
392024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
402024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
412024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
422024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
432024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
442024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
452024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Timeout
462024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
472024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
482024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
492024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
502024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
512024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
522024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
532024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
542024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
552024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
562024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
572024-12-14T04:45:06.882ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832
582024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
592024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.28\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832"}
602024-12-14T04:45:06.882ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
612024-12-14T04:45:06.882ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
622024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
632024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
642024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
652024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
662024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
672024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
682024-12-14T04:45:06.882ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 04:35:06.882573882 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
692024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
702024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
712024-12-14T04:45:06.882ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
722024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
732024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
742024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
752024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
762024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
772024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
782024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
792024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
802024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
812024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
822024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
832024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
842024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
852024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
862024-12-14T04:45:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
872024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
882024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
892024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
902024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
912024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
922024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
932024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
942024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
952024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
962024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
972024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
982024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
992024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1002024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1012024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
1022024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1032024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1042024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1052024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1062024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1072024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1082024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1092024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1102024-12-14T04:45:06.883ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
1112024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1122024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1132024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1142024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1152024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1162024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1172024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1182024-12-14T04:45:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2224041a-a9fc-46d2-b9ed-ef98dc01f696.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("229c9d57-e318-46f6-8df9-9f602a126f12.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1280, tv_nsec: 980261644 } })
1192024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1202024-12-14T04:45:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ba1572d-df24-4bd2-be03-9a99edc867d8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1280, tv_nsec: 980281934 } })
1212024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1222024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1232024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.53435ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
1242024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1252024-12-14T04:45:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.236822ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
1262024-12-14T04:45:06.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1272024-12-14T04:45:06.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1282024-12-14T04:45:06.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1292024-12-14T04:45:06.888ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1280, tv_nsec: 985317546 } })
1302024-12-14T04:45:06.888ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1312024-12-14T04:45:06.889ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1322024-12-14T04:45:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1332024-12-14T04:45:06.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1342024-12-14T04:45:06.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1352024-12-14T04:45:06.890ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362024-12-14T04:45:06.890ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.725781ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
1372024-12-14T04:45:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1382024-12-14T04:45:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1392024-12-14T04:45:06.892ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1402024-12-14T04:45:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1412024-12-14T04:45:06.894ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.047994ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1422024-12-14T04:45:06.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1432024-12-14T04:45:06.896ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1442024-12-14T04:45:06.896ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1452024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1462024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1472024-12-14T04:45:06.897ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1482024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 14.669502ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1492024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1502024-12-14T04:45:06.897ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1512024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1522024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1532024-12-14T04:45:06.897ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 04:35:06.882573882 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1542024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 15.020322ms iteration = 11 status = {"expiration":"2024-12-14T04:35:06.882573882Z","pruned":[]}
1552024-12-14T04:45:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1562024-12-14T04:45:06.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1572024-12-14T04:45:06.899ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1582024-12-14T04:45:06.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1592024-12-14T04:45:06.902ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602024-12-14T04:45:06.903ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1612024-12-14T04:45:06.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1622024-12-14T04:45:06.904ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1632024-12-14T04:45:06.904ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1642024-12-14T04:45:06.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1652024-12-14T04:45:06.907ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1281, tv_nsec: 4493075 } })
1662024-12-14T04:45:06.908ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1672024-12-14T04:45:06.908ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1682024-12-14T04:45:06.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1692024-12-14T04:45:06.913ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1702024-12-14T04:45:06.916ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1712024-12-14T04:45:06.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1722024-12-14T04:45:06.918ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1732024-12-14T04:45:06.928ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 45.484545ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
1742024-12-14T04:45:06.930ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1752024-12-14T04:45:06.930ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1762024-12-14T04:45:06.932ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1772024-12-14T04:45:06.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1782024-12-14T04:45:06.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1792024-12-14T04:45:06.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1802024-12-14T04:45:06.933ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
1812024-12-14T04:45:06.933ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
1822024-12-14T04:45:06.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1832024-12-14T04:45:06.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
1842024-12-14T04:45:06.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 54.445028ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1852024-12-14T04:45:06.936ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1862024-12-14T04:45:06.937ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1872024-12-14T04:45:06.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1882024-12-14T04:45:06.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1892024-12-14T04:45:06.940ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1902024-12-14T04:45:06.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1912024-12-14T04:45:06.943ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
1922024-12-14T04:45:06.943ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = 67c87304-f85f-490d-b105-18bacbd31c8d
1932024-12-14T04:45:06.943ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = 06ee7caa-531e-481e-b176-92cc81f5c1f7
1942024-12-14T04:45:06.943ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1952024-12-14T04:45:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1962024-12-14T04:45:06.945ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = efdc859d-ad95-4716-aa78-ca2bf8205cd5
1972024-12-14T04:45:06.945ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = 06ee7caa-531e-481e-b176-92cc81f5c1f7
1982024-12-14T04:45:06.946ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = 67c87304-f85f-490d-b105-18bacbd31c8d
1992024-12-14T04:45:06.946ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = efdc859d-ad95-4716-aa78-ca2bf8205cd5
2002024-12-14T04:45:06.946ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 63.419943ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":3,"recent_failures":[],"recent_recoveries":[]}
2012024-12-14T04:45:06.948ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2022024-12-14T04:45:06.948ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.046161ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2032024-12-14T04:45:06.950ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2042024-12-14T04:45:06.951ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2052024-12-14T04:45:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2062024-12-14T04:45:06.953ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2072024-12-14T04:45:06.953ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2082024-12-14T04:45:06.953ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
2092024-12-14T04:45:06.953ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 71.444601ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2102024-12-14T04:45:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2112024-12-14T04:45:06.955ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2122024-12-14T04:45:06.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2132024-12-14T04:45:06.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2142024-12-14T04:45:06.957ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2152024-12-14T04:45:06.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2162024-12-14T04:45:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2172024-12-14T04:45:06.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2182024-12-14T04:45:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2192024-12-14T04:45:06.961ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 79.243335ms iteration = 21 status = {"errors":[],"records_set_to_done":[]}
2202024-12-14T04:45:06.964ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2212024-12-14T04:45:06.966ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2222024-12-14T04:45:06.966ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2232024-12-14T04:45:06.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2242024-12-14T04:45:06.967ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2252024-12-14T04:45:06.967ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2262024-12-14T04:45:06.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2272024-12-14T04:45:06.970ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2282024-12-14T04:45:06.970ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2292024-12-14T04:45:06.970ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2302024-12-14T04:45:06.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2312024-12-14T04:45:06.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2322024-12-14T04:45:06.980ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2332024-12-14T04:45:06.980ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2342024-12-14T04:45:06.980ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2352024-12-14T04:45:06.980ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2362024-12-14T04:45:06.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2372024-12-14T04:45:06.990ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2382024-12-14T04:45:06.994ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2392024-12-14T04:45:06.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2402024-12-14T04:45:06.996ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2412024-12-14T04:45:06.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2422024-12-14T04:45:07.001ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2432024-12-14T04:45:07.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2442024-12-14T04:45:07.003ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2452024-12-14T04:45:07.003ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2462024-12-14T04:45:07.003ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2472024-12-14T04:45:07.003ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2482024-12-14T04:45:07.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2492024-12-14T04:45:07.036ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2502024-12-14T04:45:07.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2512024-12-14T04:45:07.045ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2522024-12-14T04:45:07.045ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2532024-12-14T04:45:07.045ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bd9038d-385f-41e6-956b-913f6e761901", "content-length": "26", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
2542024-12-14T04:45:07.045ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2552024-12-14T04:45:07.046ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2562024-12-14T04:45:07.072ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2572024-12-14T04:45:07.077ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2582024-12-14T04:45:07.077ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2592024-12-14T04:45:07.077ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2602024-12-14T04:45:07.078ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "da2d3934-4ed5-4adb-9d39-5e031c02af43", "content-length": "372", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
2612024-12-14T04:45:07.078ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2622024-12-14T04:45:07.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2632024-12-14T04:45:07.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2642024-12-14T04:45:07.080ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1281, tv_nsec: 177603706 } })
2652024-12-14T04:45:07.080ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2662024-12-14T04:45:07.080ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
2672024-12-14T04:45:07.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2682024-12-14T04:45:07.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2692024-12-14T04:45:07.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2702024-12-14T04:45:07.082ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2712024-12-14T04:45:07.084ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(8e7aed19-dfbf-448e-9e63-dce2f2ac3890, 0), (16577fe2-edab-45b8-998d-19604bed377f, 0), (fba3df1a-2d2e-425a-a329-95ba4eb3ae96, 0), (ca5678e2-009e-46e5-9e96-fe82d94f7247, 0)] collection_id = 8e7aed19-dfbf-448e-9e63-dce2f2ac3890
2722024-12-14T04:45:07.084ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2732024-12-14T04:45:07.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2742024-12-14T04:45:07.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2752024-12-14T04:45:07.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2762024-12-14T04:45:07.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2772024-12-14T04:45:07.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2782024-12-14T04:45:07.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2792024-12-14T04:45:07.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2802024-12-14T04:45:07.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2812024-12-14T04:45:07.126ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2822024-12-14T04:45:07.128ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2832024-12-14T04:45:07.128ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 245.950259ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2842024-12-14T04:45:07.129ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 246.630016ms iteration = 21 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2852024-12-14T04:45:07.130ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2862024-12-14T04:45:07.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2872024-12-14T04:45:07.133ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2882024-12-14T04:45:07.136ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2892024-12-14T04:45:07.136ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2902024-12-14T04:45:07.137ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2912024-12-14T04:45:07.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2922024-12-14T04:45:07.138ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2932024-12-14T04:45:07.139ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 8e7aed19-dfbf-448e-9e63-dce2f2ac3890 file = nexus/db-queries/src/db/datastore/inventory.rs:1442 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 0 nerrors = 0 nnics = 0 nnvme_disk_firmware = 0 nphysical_disks = 0 nrot_pages = 8 nrots = 2 nsled_agent_zones = 0 nsled_agents = 0 nsps = 2 nzones = 0 nzpools = 0
2942024-12-14T04:45:07.139ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2952024-12-14T04:45:07.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
2962024-12-14T04:45:07.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2972024-12-14T04:45:07.140ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 257.979942ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2982024-12-14T04:45:07.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2992024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3002024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3012024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3022024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3032024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3042024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3052024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3062024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3072024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3082024-12-14T04:45:07.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3092024-12-14T04:45:07.142ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3102024-12-14T04:45:07.142ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3112024-12-14T04:45:07.142ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 259.678776ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3122024-12-14T04:45:07.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3132024-12-14T04:45:07.144ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3142024-12-14T04:45:07.144ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3152024-12-14T04:45:07.144ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 0 version = 2
3162024-12-14T04:45:07.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3172024-12-14T04:45:07.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3182024-12-14T04:45:07.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3192024-12-14T04:45:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3202024-12-14T04:45:07.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3212024-12-14T04:45:07.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3222024-12-14T04:45:07.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3232024-12-14T04:45:07.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3242024-12-14T04:45:07.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3252024-12-14T04:45:07.146ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(16577fe2-edab-45b8-998d-19604bed377f, 0), (fba3df1a-2d2e-425a-a329-95ba4eb3ae96, 0), (ca5678e2-009e-46e5-9e96-fe82d94f7247, 0)]
3262024-12-14T04:45:07.147ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1281, tv_nsec: 244388136 } })
3272024-12-14T04:45:07.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
3282024-12-14T04:45:07.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3292024-12-14T04:45:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3302024-12-14T04:45:07.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3312024-12-14T04:45:07.153ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3322024-12-14T04:45:07.153ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3332024-12-14T04:45:07.153ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 04:35:18.025163 UTC dns_group = internal generation = 1
3342024-12-14T04:45:07.153ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 270.733905ms iteration = 12 status = {"generation":1}
3352024-12-14T04:45:07.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3362024-12-14T04:45:07.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3372024-12-14T04:45:07.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3382024-12-14T04:45:07.155ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 1 version = 2
3392024-12-14T04:45:07.155ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3402024-12-14T04:45:07.155ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 04:35:19.721992 UTC dns_group = external generation = 2
3412024-12-14T04:45:07.155ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 273.433115ms iteration = 12 status = {"generation":2}
3422024-12-14T04:45:07.163ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3432024-12-14T04:45:07.163ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3442024-12-14T04:45:07.166ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b7abfce9-8b43-496e-8101-5adc9f0fccce", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
3452024-12-14T04:45:07.166ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3462024-12-14T04:45:07.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3472024-12-14T04:45:07.166ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
3482024-12-14T04:45:07.166ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 283.886028ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
3492024-12-14T04:45:07.168ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3502024-12-14T04:45:07.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3512024-12-14T04:45:07.171ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3522024-12-14T04:45:07.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3532024-12-14T04:45:07.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3542024-12-14T04:45:07.175ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3552024-12-14T04:45:07.175ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3562024-12-14T04:45:07.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
3572024-12-14T04:45:07.176ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3582024-12-14T04:45:07.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3592024-12-14T04:45:07.179ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "05d8f499-cd4d-49b1-aaf9-cda49911e4c8", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
3602024-12-14T04:45:07.179ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 296.875335ms iteration = 21 status = 0
3612024-12-14T04:45:07.180ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3622024-12-14T04:45:07.180ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 298.386404ms iteration = 22 status = {}
3632024-12-14T04:45:07.181ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T04:35:03.711237Z, time_modified: 2024-12-14T04:35:03.711237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }) }
3642024-12-14T04:45:07.181ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 298.98031ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]}},"default_endpoint":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3652024-12-14T04:45:07.183ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3662024-12-14T04:45:07.183ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3672024-12-14T04:45:07.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3682024-12-14T04:45:07.202ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3692024-12-14T04:45:07.202ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3702024-12-14T04:45:07.202ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b509346-dc5d-4c11-b56a-4b74907cc978", "content-length": "26", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
3712024-12-14T04:45:07.202ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3722024-12-14T04:45:07.204ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3732024-12-14T04:45:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 326.122567ms iteration = 21 status = {}
3742024-12-14T04:45:07.210ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3752024-12-14T04:45:07.210ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 327.903274ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3762024-12-14T04:45:07.216ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3772024-12-14T04:45:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3782024-12-14T04:45:07.218ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3792024-12-14T04:45:07.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3802024-12-14T04:45:07.220ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3812024-12-14T04:45:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3822024-12-14T04:45:07.221ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3832024-12-14T04:45:07.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3842024-12-14T04:45:07.223ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3852024-12-14T04:45:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3862024-12-14T04:45:07.225ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
3872024-12-14T04:45:07.225ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3882024-12-14T04:45:07.225ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f270ba7-c5f9-4f50-b8f9-8d3ea518983d", "content-length": "26", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
3892024-12-14T04:45:07.225ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3902024-12-14T04:45:07.227ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
3912024-12-14T04:45:07.249ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3922024-12-14T04:45:07.250ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 367.434484ms iteration = 24 status = {"physical_disks_added":0}
3932024-12-14T04:45:07.250ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a7ef19f7-8d48-4f57-9a3b-267d54f72187", "date": "Sat, 14 Dec 2024 04:45:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3942024-12-14T04:45:07.250ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3952024-12-14T04:45:07.251ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3cbbaa9d-790a-4111-832e-027ee15de7f5", "date": "Sat, 14 Dec 2024 04:45:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3962024-12-14T04:45:07.251ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3972024-12-14T04:45:07.252ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e6914b7c-2b63-4ccb-bd12-c6a6a534c110", "date": "Sat, 14 Dec 2024 04:45:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3982024-12-14T04:45:07.252ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3992024-12-14T04:45:07.254ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4002024-12-14T04:45:07.254ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 372.334191ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4012024-12-14T04:45:07.258ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4022024-12-14T04:45:07.259ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f6095d85-2fa2-437c-92f5-9aacd371fabe", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4032024-12-14T04:45:07.259ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4042024-12-14T04:45:07.261ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "29934de4-32aa-4698-a81b-278c84bab6d2", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4052024-12-14T04:45:07.261ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4062024-12-14T04:45:07.263ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4072024-12-14T04:45:07.263ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 380.668489ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4082024-12-14T04:45:07.277ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
4092024-12-14T04:45:07.277ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
4102024-12-14T04:45:07.277ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
4112024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3914d0eb-c640-4473-89db-3833c80203ca", "content-length": "476", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4122024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
4132024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7927f465-870a-47f1-b329-f453518c087e", "content-length": "734", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4142024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
4152024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "90ff4335-9d3f-4f84-978f-7a448083f6ac", "content-length": "111", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4162024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
4172024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d10904f-6c25-4e8b-b023-2d79986bbe2c", "content-length": "111", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4182024-12-14T04:45:07.278ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
4192024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ff0792f-7cd5-4007-aea7-4a9a285c94c8", "content-length": "167", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4202024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
4212024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f83bbc07-93a7-419b-b336-1c4cb1a32593", "content-length": "167", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4222024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
4232024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8244648c-7b1f-4ffd-ba38-37364089c79b", "content-length": "175", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4242024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
4252024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6de90f2a-af60-421c-bc58-a906eee1a5f0", "content-length": "175", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4262024-12-14T04:45:07.279ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
4272024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36983a8f-290b-456f-8c89-f3d333a2a0c7", "content-length": "702", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4282024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
4292024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03717aff-8aea-4cc1-8ade-321af29a2468", "content-length": "727", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4302024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
4312024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a30f5ddc-b863-4587-b228-7d3018b75cc1", "content-length": "729", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4322024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
4332024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd378564-f72f-4f2c-a188-58b9a2d73aea", "content-length": "728", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4342024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
4352024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b01c838c-5289-47ef-8e22-91930da7985c", "content-length": "722", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4362024-12-14T04:45:07.280ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
4372024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21391605-845d-485c-8248-e35c01197ac8", "content-length": "109", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4382024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
4392024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd2881ee-0c89-4fb1-9beb-2b28d079ebc5", "content-length": "109", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4402024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
4412024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9689274f-21e4-444b-9b1e-49006517f698", "content-length": "169", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4422024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
4432024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66c15689-592f-414d-bd80-ac3d27de94ff", "content-length": "169", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4442024-12-14T04:45:07.281ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
4452024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d2ff115-1f5d-4f5e-bce2-123f75a8b957", "content-length": "178", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4462024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
4472024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8bface4-d17a-47fa-8d99-2d0cca809b6b", "content-length": "178", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4482024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
4492024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "951065a2-1061-4b21-8d52-3111ff918034", "content-length": "702", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4502024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
4512024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7473f9e3-8c6b-46aa-8411-70b2ee7779d5", "content-length": "727", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4522024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
4532024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56e7cab7-5a9d-447d-bc37-8563f96a5d63", "content-length": "729", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4542024-12-14T04:45:07.282ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
4552024-12-14T04:45:07.283ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5c4d6ae-11cd-44d2-8da9-ff303349a3ed", "content-length": "728", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4562024-12-14T04:45:07.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4572024-12-14T04:45:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4582024-12-14T04:45:07.284ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4592024-12-14T04:45:07.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4602024-12-14T04:45:07.285ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c56a4fbd-2d1c-42f3-93c2-07f49a8403a7", "content-length": "1", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4612024-12-14T04:45:07.286ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4622024-12-14T04:45:07.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
4632024-12-14T04:45:07.286ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4642024-12-14T04:45:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4652024-12-14T04:45:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4662024-12-14T04:45:07.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4672024-12-14T04:45:07.288ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 406.43356ms iteration = 21 status = null
4682024-12-14T04:45:07.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
4692024-12-14T04:45:07.289ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4702024-12-14T04:45:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4712024-12-14T04:45:07.304ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4722024-12-14T04:45:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4732024-12-14T04:45:07.354ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4742024-12-14T04:45:07.355ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1275a65-ff77-40ae-ba20-4645892ba2fc", "content-length": "691", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4752024-12-14T04:45:07.355ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
4762024-12-14T04:45:07.355ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4772024-12-14T04:45:07.355ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4782024-12-14T04:45:07.357ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4792024-12-14T04:45:07.357ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 474.72111ms iteration = 21 status = {}
4802024-12-14T04:45:07.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4812024-12-14T04:45:07.358ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4822024-12-14T04:45:07.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4832024-12-14T04:45:07.361ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4842024-12-14T04:45:07.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4852024-12-14T04:45:07.390ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
4862024-12-14T04:45:07.390ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
4872024-12-14T04:45:07.392ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4882024-12-14T04:45:07.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4892024-12-14T04:45:07.394ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
4902024-12-14T04:45:07.394ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4912024-12-14T04:45:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4922024-12-14T04:45:07.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 84dbd85c-2ecf-4cf6-9673-c5f004f50175 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
4932024-12-14T04:45:07.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 84dbd85c-2ecf-4cf6-9673-c5f004f50175 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
4942024-12-14T04:45:07.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1934 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 84dbd85c-2ecf-4cf6-9673-c5f004f50175 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
4952024-12-14T04:45:07.405ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4962024-12-14T04:45:07.406ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ac4ebf-0972-4117-9a07-f70499b0c769", "content-length": "134", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
4972024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4982024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
4992024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5002024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5012024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5022024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5032024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5042024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5052024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5062024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5072024-12-14T04:45:07.406ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
5082024-12-14T04:45:07.406ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5092024-12-14T04:45:07.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5102024-12-14T04:45:07.409ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5112024-12-14T04:45:07.409ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9686fb9-da0f-4afb-a0d5-af3635743e2a", "content-length": "2", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
5122024-12-14T04:45:07.409ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5132024-12-14T04:45:07.409ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0de1f779-5511-4098-a345-2497abe6a67a", "content-length": "52", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
5142024-12-14T04:45:07.409ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
5152024-12-14T04:45:07.409ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5162024-12-14T04:45:07.420ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5172024-12-14T04:45:07.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5182024-12-14T04:45:07.422ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5192024-12-14T04:45:07.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5202024-12-14T04:45:07.424ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5212024-12-14T04:45:07.427ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5222024-12-14T04:45:07.427ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5232024-12-14T04:45:07.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5242024-12-14T04:45:07.429ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5252024-12-14T04:45:07.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5262024-12-14T04:45:07.431ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5272024-12-14T04:45:07.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5282024-12-14T04:45:07.434ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5292024-12-14T04:45:07.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5302024-12-14T04:45:07.439ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5312024-12-14T04:45:07.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5322024-12-14T04:45:07.442ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5332024-12-14T04:45:07.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5342024-12-14T04:45:07.444ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5352024-12-14T04:45:07.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5362024-12-14T04:45:07.446ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5372024-12-14T04:45:07.450ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
5382024-12-14T04:45:07.450ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5392024-12-14T04:45:07.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5402024-12-14T04:45:07.451ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5412024-12-14T04:45:07.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5422024-12-14T04:45:07.453ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5432024-12-14T04:45:07.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5442024-12-14T04:45:07.454ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452024-12-14T04:45:07.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
5462024-12-14T04:45:07.458ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5472024-12-14T04:45:07.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5482024-12-14T04:45:07.462ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5492024-12-14T04:45:07.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5502024-12-14T04:45:07.464ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5512024-12-14T04:45:07.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5522024-12-14T04:45:07.466ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5532024-12-14T04:45:07.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5542024-12-14T04:45:07.468ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5552024-12-14T04:45:07.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5562024-12-14T04:45:07.471ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5572024-12-14T04:45:07.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5582024-12-14T04:45:07.472ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5592024-12-14T04:45:07.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5602024-12-14T04:45:07.474ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5612024-12-14T04:45:07.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5622024-12-14T04:45:07.475ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5632024-12-14T04:45:07.479ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
5642024-12-14T04:45:07.479ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5652024-12-14T04:45:07.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5662024-12-14T04:45:07.480ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5672024-12-14T04:45:07.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5682024-12-14T04:45:07.481ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5692024-12-14T04:45:07.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5702024-12-14T04:45:07.483ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5712024-12-14T04:45:07.486ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
5722024-12-14T04:45:07.486ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5732024-12-14T04:45:07.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5742024-12-14T04:45:07.492ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 884c1dd6-beb8-4af4-a5a6-31995ecbb183, kind: Service { id: bb43c8e4-f24f-424b-aa30-a7b5d215708e }, name: Name("nexus-bb43c8e4-f24f-424b-aa30-a7b5d215708e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 185])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 985a6f50-eb4b-450e-8994-8e677143cd87, kind: Service { id: e36fd091-efce-4ea5-9e73-a9bc4cb7a549 }, name: Name("nexus-e36fd091-efce-4ea5-9e73-a9bc4cb7a549"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 234])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f0e8ac24-f32d-4488-9c3c-9af00b9bda6f, kind: Service { id: a2394d1d-754d-4ed4-8218-d02ec0d792df }, name: Name("nexus-a2394d1d-754d-4ed4-8218-d02ec0d792df"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 95])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 0d2b4f8f-354d-4398-8d3a-37472bac15e3, kind: Service { id: b0cf818e-eb68-4273-b879-3df82709e7d6 }, name: Name("external-dns-b0cf818e-eb68-4273-b879-3df82709e7d6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 149])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 749b346b-1343-4e3a-a01e-e19ce433ac08, kind: Service { id: 6ba1572d-df24-4bd2-be03-9a99edc867d8 }, name: Name("external-dns-6ba1572d-df24-4bd2-be03-9a99edc867d8"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
5752024-12-14T04:45:07.492ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5762024-12-14T04:45:07.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5772024-12-14T04:45:07.493ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782024-12-14T04:45:07.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
5792024-12-14T04:45:07.495ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5802024-12-14T04:45:07.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5812024-12-14T04:45:07.497ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
5822024-12-14T04:45:07.497ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 0d2b4f8f-354d-4398-8d3a-37472bac15e3, kind: Service { id: b0cf818e-eb68-4273-b879-3df82709e7d6 }, name: Name("external-dns-b0cf818e-eb68-4273-b879-3df82709e7d6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 149])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 749b346b-1343-4e3a-a01e-e19ce433ac08, kind: Service { id: 6ba1572d-df24-4bd2-be03-9a99edc867d8 }, name: Name("external-dns-6ba1572d-df24-4bd2-be03-9a99edc867d8"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 884c1dd6-beb8-4af4-a5a6-31995ecbb183, kind: Service { id: bb43c8e4-f24f-424b-aa30-a7b5d215708e }, name: Name("nexus-bb43c8e4-f24f-424b-aa30-a7b5d215708e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 185])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 985a6f50-eb4b-450e-8994-8e677143cd87, kind: Service { id: e36fd091-efce-4ea5-9e73-a9bc4cb7a549 }, name: Name("nexus-e36fd091-efce-4ea5-9e73-a9bc4cb7a549"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 234])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f0e8ac24-f32d-4488-9c3c-9af00b9bda6f, kind: Service { id: a2394d1d-754d-4ed4-8218-d02ec0d792df }, name: Name("nexus-a2394d1d-754d-4ed4-8218-d02ec0d792df"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 95])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
5832024-12-14T04:45:07.497ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
5842024-12-14T04:45:07.535ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ffea1404-561b-402c-bfb5-0fbe03f25482", "date": "Sat, 14 Dec 2024 04:45:07 GMT"} })
5852024-12-14T04:45:07.535ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
5862024-12-14T04:45:07.535ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5872024-12-14T04:45:07.535ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "24da9cee-c46b-4f7b-bd3b-1ada0d8693ac", "content-length": "682", "date": "Sat, 14 Dec 2024 04:45:06 GMT"} })
5882024-12-14T04:45:07.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5892024-12-14T04:45:07.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5902024-12-14T04:45:07.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5912024-12-14T04:45:07.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5922024-12-14T04:45:07.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5932024-12-14T04:45:07.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5942024-12-14T04:45:07.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5952024-12-14T04:45:07.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5962024-12-14T04:45:07.553ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 670.363927ms iteration = 21 status = {}
5972024-12-14T04:45:07.613ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, time_created: 2024-12-14T04:35:18.598624Z, time_modified: 2024-12-14T04:35:18.598624Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
5982024-12-14T04:45:07.613ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
5992024-12-14T04:45:07.613ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6002024-12-14T04:45:07.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6012024-12-14T04:45:07.615ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6022024-12-14T04:45:07.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6032024-12-14T04:45:07.617ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6042024-12-14T04:45:07.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
6052024-12-14T04:45:07.636ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3898f06a-ed3a-498d-962f-3c6803086315", "content-length": "30429", "date": "Sat, 14 Dec 2024 04:45:07 GMT"} })
6062024-12-14T04:45:07.639ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
6072024-12-14T04:45:07.639ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
6082024-12-14T04:45:07.639ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
6092024-12-14T04:45:07.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6102024-12-14T04:45:07.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6112024-12-14T04:45:07.642ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6122024-12-14T04:45:07.644ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
6132024-12-14T04:45:07.644ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6142024-12-14T04:45:07.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6152024-12-14T04:45:07.650ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
6162024-12-14T04:45:07.654ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "4b6a9bd9-9fe6-4f0a-9a24-5f55bbe4aceb", "date": "Sat, 14 Dec 2024 04:45:07 GMT"} })
6172024-12-14T04:45:07.655ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
6182024-12-14T04:45:07.655ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 773.381242ms
6192024-12-14T04:45:07.655ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 773.395762ms iteration = 3 status = {}
6202024-12-14T04:45:07.812ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = cd40dc24-d622-4271-b99d-50b34f04ef87 file = nexus/db-queries/src/db/datastore/inventory.rs:1010
6212024-12-14T04:45:07.812ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = cd40dc24-d622-4271-b99d-50b34f04ef87 time_started = 2024-12-14 04:45:07.277595 UTC
6222024-12-14T04:45:07.812ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 929.842639ms iteration = 4 status = {"collection_id":"cd40dc24-d622-4271-b99d-50b34f04ef87","time_done":"2024-12-14 04:45:07.639015 UTC","time_started":"2024-12-14 04:45:07.277595 UTC"}
6232024-12-14T04:45:07.812ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Dependency
6242024-12-14T04:45:07.812ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6252024-12-14T04:45:07.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6262024-12-14T04:45:07.814ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6272024-12-14T04:45:07.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6282024-12-14T04:45:07.816ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6292024-12-14T04:45:07.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6302024-12-14T04:45:07.818ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6312024-12-14T04:45:07.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6322024-12-14T04:45:07.820ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6332024-12-14T04:45:07.821ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6342024-12-14T04:45:07.821ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6352024-12-14T04:45:07.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6362024-12-14T04:45:07.904ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 91.968126ms iteration = 25 status = {"physical_disks_added":0}
6372024-12-14T04:45:08.424ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2c2819cb-3f99-4cb0-8ba5-9bbcf2db948f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6382024-12-14T04:45:08.425ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2c2819cb-3f99-4cb0-8ba5-9bbcf2db948f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6392024-12-14T04:45:08.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2170 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2c2819cb-3f99-4cb0-8ba5-9bbcf2db948f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6402024-12-14T04:45:09.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e00544f7-efc1-40f6-9461-610d13847b13 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6412024-12-14T04:45:09.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e00544f7-efc1-40f6-9461-610d13847b13 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6422024-12-14T04:45:09.448ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2493 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e00544f7-efc1-40f6-9461-610d13847b13 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6432024-12-14T04:45:10.467ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 81e5167a-3e5c-4b31-9f85-f91f0fb00699 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6442024-12-14T04:45:10.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 81e5167a-3e5c-4b31-9f85-f91f0fb00699 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6452024-12-14T04:45:10.469ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2202 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 81e5167a-3e5c-4b31-9f85-f91f0fb00699 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6462024-12-14T04:45:11.490ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8ff2a070-1187-4154-a076-3680d4e1f66a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6472024-12-14T04:45:11.490ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8ff2a070-1187-4154-a076-3680d4e1f66a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6482024-12-14T04:45:11.492ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1963 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8ff2a070-1187-4154-a076-3680d4e1f66a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6492024-12-14T04:45:12.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = deb22add-b0a6-443e-925a-6545f1bc2f69 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6502024-12-14T04:45:12.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = deb22add-b0a6-443e-925a-6545f1bc2f69 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6512024-12-14T04:45:12.515ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2709 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = deb22add-b0a6-443e-925a-6545f1bc2f69 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6522024-12-14T04:45:12.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1244 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 0215691e-d62e-41c1-ad8d-0844ad0cabf3 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
6532024-12-14T04:45:13.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2c00ac41-3c3e-41a7-8056-21d36f001548 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6542024-12-14T04:45:13.535ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2c00ac41-3c3e-41a7-8056-21d36f001548 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6552024-12-14T04:45:13.536ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1842 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2c00ac41-3c3e-41a7-8056-21d36f001548 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6562024-12-14T04:45:14.556ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83beacdf-73a2-49b4-a049-13838a636808 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6572024-12-14T04:45:14.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83beacdf-73a2-49b4-a049-13838a636808 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6582024-12-14T04:45:14.558ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1824 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83beacdf-73a2-49b4-a049-13838a636808 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6592024-12-14T04:45:15.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 483877c6-b438-45bd-8626-918b9f21ddd4 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6602024-12-14T04:45:15.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 483877c6-b438-45bd-8626-918b9f21ddd4 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6612024-12-14T04:45:15.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1968 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 483877c6-b438-45bd-8626-918b9f21ddd4 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6622024-12-14T04:45:16.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca9f9b37-f615-4382-9bbf-a43465d1422b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6632024-12-14T04:45:16.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca9f9b37-f615-4382-9bbf-a43465d1422b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6642024-12-14T04:45:16.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2185 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca9f9b37-f615-4382-9bbf-a43465d1422b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6652024-12-14T04:45:16.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
6662024-12-14T04:45:16.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6672024-12-14T04:45:16.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6682024-12-14T04:45:16.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
6692024-12-14T04:45:16.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6702024-12-14T04:45:16.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6712024-12-14T04:45:16.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6722024-12-14T04:45:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6732024-12-14T04:45:16.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6742024-12-14T04:45:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6752024-12-14T04:45:16.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6762024-12-14T04:45:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6772024-12-14T04:45:16.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782024-12-14T04:45:16.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
6792024-12-14T04:45:16.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6802024-12-14T04:45:16.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6812024-12-14T04:45:16.901ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
6822024-12-14T04:45:16.901ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.625215ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
6832024-12-14T04:45:16.909ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6842024-12-14T04:45:16.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6852024-12-14T04:45:16.942ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.001513ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6862024-12-14T04:45:17.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fc140ad8-6b08-4cd6-ba1a-8fbd1b5db2c2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6872024-12-14T04:45:17.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fc140ad8-6b08-4cd6-ba1a-8fbd1b5db2c2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6882024-12-14T04:45:17.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2346 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fc140ad8-6b08-4cd6-ba1a-8fbd1b5db2c2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6892024-12-14T04:45:18.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8fee8b53-ec21-49e5-9162-d5b0b8f25bbb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6902024-12-14T04:45:18.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8fee8b53-ec21-49e5-9162-d5b0b8f25bbb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6912024-12-14T04:45:18.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1807 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8fee8b53-ec21-49e5-9162-d5b0b8f25bbb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6922024-12-14T04:45:19.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0458403c-e854-484b-99ed-aab992d0dc41 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6932024-12-14T04:45:19.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0458403c-e854-484b-99ed-aab992d0dc41 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6942024-12-14T04:45:19.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1747 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0458403c-e854-484b-99ed-aab992d0dc41 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
6952024-12-14T04:45:20.191ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:61512
6962024-12-14T04:45:20.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:61512 req_id = 8734852c-cf01-4137-be89-23b423878ad1 roles = RoleSet { roles: {} } uri = /metrics/producers
6972024-12-14T04:45:20.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:61512 req_id = 8734852c-cf01-4137-be89-23b423878ad1 resource = Database result = Ok(()) uri = /metrics/producers
6982024-12-14T04:45:20.236ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 5f4909e4-dcea-4db6-99c3-a64609b9f774
6992024-12-14T04:45:20.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45012 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:61512 req_id = 8734852c-cf01-4137-be89-23b423878ad1 response_code = 201 uri = /metrics/producers
7002024-12-14T04:45:20.686ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e0213fa7-562e-4a2f-990d-44ab9d63d322 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7012024-12-14T04:45:20.687ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e0213fa7-562e-4a2f-990d-44ab9d63d322 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7022024-12-14T04:45:20.688ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2290 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e0213fa7-562e-4a2f-990d-44ab9d63d322 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7032024-12-14T04:45:21.709ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 70995f4e-a02e-45e4-99ad-9aa25884670c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7042024-12-14T04:45:21.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 70995f4e-a02e-45e4-99ad-9aa25884670c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7052024-12-14T04:45:21.711ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2123 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 70995f4e-a02e-45e4-99ad-9aa25884670c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7062024-12-14T04:45:22.301ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
7072024-12-14T04:45:22.301ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
7082024-12-14T04:45:22.332ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
7092024-12-14T04:45:22.332ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:47314
7102024-12-14T04:45:22.333ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47314 req_id = ed5f1e90-5969-4573-886b-964d968761e4 roles = RoleSet { roles: {} } uri = /metrics/producers
7112024-12-14T04:45:22.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47314 req_id = ed5f1e90-5969-4573-886b-964d968761e4 resource = Database result = Ok(()) uri = /metrics/producers
7122024-12-14T04:45:22.378ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
7132024-12-14T04:45:22.378ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45327 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47314 req_id = ed5f1e90-5969-4573-886b-964d968761e4 response_code = 201 uri = /metrics/producers
7142024-12-14T04:45:22.378ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "ed5f1e90-5969-4573-886b-964d968761e4", "content-length": "41", "date": "Sat, 14 Dec 2024 04:45:21 GMT"} })
7152024-12-14T04:45:22.380ZDEBGnexus (producer-registration-task): registered with nexus successfully
7162024-12-14T04:45:22.380ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
7172024-12-14T04:45:22.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c93dcb1a-f625-4d29-be8c-cf720d0952ec roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7182024-12-14T04:45:22.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c93dcb1a-f625-4d29-be8c-cf720d0952ec resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7192024-12-14T04:45:22.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1933 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c93dcb1a-f625-4d29-be8c-cf720d0952ec response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7202024-12-14T04:45:22.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1299 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 82ecb0db-20a3-4e9f-8631-5846664816d0 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
7212024-12-14T04:45:23.757ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 92603924-82e1-41e4-92f0-8f6666410b42 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7222024-12-14T04:45:23.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 92603924-82e1-41e4-92f0-8f6666410b42 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7232024-12-14T04:45:23.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2543 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 92603924-82e1-41e4-92f0-8f6666410b42 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7242024-12-14T04:45:24.135ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:50809
7252024-12-14T04:45:24.135ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:50809 req_id = 6b135264-327e-4037-920e-f04f91bd8417 roles = RoleSet { roles: {} } uri = /metrics/producers
7262024-12-14T04:45:24.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:50809 req_id = 6b135264-327e-4037-920e-f04f91bd8417 resource = Database result = Ok(()) uri = /metrics/producers
7272024-12-14T04:45:24.180ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = b1576088-d04d-48cf-a285-75e4628d77d5
7282024-12-14T04:45:24.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44274 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:50809 req_id = 6b135264-327e-4037-920e-f04f91bd8417 response_code = 201 uri = /metrics/producers
7292024-12-14T04:45:24.781ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 15ebe715-92c9-45bd-889a-3958f53e2be7 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7302024-12-14T04:45:24.781ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 15ebe715-92c9-45bd-889a-3958f53e2be7 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7312024-12-14T04:45:24.783ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2445 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 15ebe715-92c9-45bd-889a-3958f53e2be7 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7322024-12-14T04:45:25.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8b3f06f-b913-4105-a7c9-5a3a4eae9357 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7332024-12-14T04:45:25.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8b3f06f-b913-4105-a7c9-5a3a4eae9357 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7342024-12-14T04:45:25.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1881 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8b3f06f-b913-4105-a7c9-5a3a4eae9357 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7352024-12-14T04:45:26.464ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:54571
7362024-12-14T04:45:26.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54571 req_id = 12c01ba4-455f-45b1-97fd-260bce4a42a6 roles = RoleSet { roles: {} } uri = /metrics/producers
7372024-12-14T04:45:26.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54571 req_id = 12c01ba4-455f-45b1-97fd-260bce4a42a6 resource = Database result = Ok(()) uri = /metrics/producers
7382024-12-14T04:45:26.500ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 83153bbf-d864-44c8-b8a5-770b4235fe7a
7392024-12-14T04:45:26.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35913 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54571 req_id = 12c01ba4-455f-45b1-97fd-260bce4a42a6 response_code = 201 uri = /metrics/producers
7402024-12-14T04:45:26.826ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c943168b-798e-4e54-bd53-ac2ae6b99d4e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7412024-12-14T04:45:26.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c943168b-798e-4e54-bd53-ac2ae6b99d4e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7422024-12-14T04:45:26.830ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3882 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c943168b-798e-4e54-bd53-ac2ae6b99d4e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7432024-12-14T04:45:26.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
7442024-12-14T04:45:26.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7452024-12-14T04:45:26.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7462024-12-14T04:45:26.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
7472024-12-14T04:45:26.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7482024-12-14T04:45:26.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7492024-12-14T04:45:26.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7502024-12-14T04:45:26.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7512024-12-14T04:45:26.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7522024-12-14T04:45:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7532024-12-14T04:45:26.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7542024-12-14T04:45:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7552024-12-14T04:45:26.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7562024-12-14T04:45:26.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7572024-12-14T04:45:26.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7582024-12-14T04:45:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
7592024-12-14T04:45:26.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7602024-12-14T04:45:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7612024-12-14T04:45:26.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
7622024-12-14T04:45:26.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.232595ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
7632024-12-14T04:45:26.913ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.164032ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7642024-12-14T04:45:27.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 08e762ef-87d3-4899-a303-e9844976663b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7652024-12-14T04:45:27.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 08e762ef-87d3-4899-a303-e9844976663b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7662024-12-14T04:45:27.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1829 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 08e762ef-87d3-4899-a303-e9844976663b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7672024-12-14T04:45:28.880ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3f97ae6b-6bf5-426f-bd5a-c4e6fb7bf8fd roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7682024-12-14T04:45:28.880ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3f97ae6b-6bf5-426f-bd5a-c4e6fb7bf8fd resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7692024-12-14T04:45:28.882ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2146 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3f97ae6b-6bf5-426f-bd5a-c4e6fb7bf8fd response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7702024-12-14T04:45:29.902ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3f9d9be5-d13f-40f8-ad8b-3c076749fd93 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7712024-12-14T04:45:29.902ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3f9d9be5-d13f-40f8-ad8b-3c076749fd93 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7722024-12-14T04:45:29.904ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1726 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3f9d9be5-d13f-40f8-ad8b-3c076749fd93 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7732024-12-14T04:45:30.924ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37c5c8ea-a62b-42ba-92bc-63a625fb4c39 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7742024-12-14T04:45:30.925ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37c5c8ea-a62b-42ba-92bc-63a625fb4c39 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7752024-12-14T04:45:30.927ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2849 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37c5c8ea-a62b-42ba-92bc-63a625fb4c39 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7762024-12-14T04:45:31.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7738103b-4c2f-4ba6-9c28-1a121f09c70d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7772024-12-14T04:45:31.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7738103b-4c2f-4ba6-9c28-1a121f09c70d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7782024-12-14T04:45:31.950ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2017 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7738103b-4c2f-4ba6-9c28-1a121f09c70d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7792024-12-14T04:45:32.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1417 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 0cc149e3-f551-42d1-a0f8-a0e61a82ca38 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
7802024-12-14T04:45:32.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3507f32e-fae9-4a1e-8d8e-289c5ac791ff roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7812024-12-14T04:45:32.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3507f32e-fae9-4a1e-8d8e-289c5ac791ff resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7822024-12-14T04:45:32.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2345 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3507f32e-fae9-4a1e-8d8e-289c5ac791ff response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7832024-12-14T04:45:33.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3d8dcb25-b8f1-4762-990c-8f8224ac1987 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7842024-12-14T04:45:33.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3d8dcb25-b8f1-4762-990c-8f8224ac1987 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7852024-12-14T04:45:33.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2506 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3d8dcb25-b8f1-4762-990c-8f8224ac1987 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7862024-12-14T04:45:35.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5c0c8168-b73f-4d1d-96d9-d9105b58580b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7872024-12-14T04:45:35.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5c0c8168-b73f-4d1d-96d9-d9105b58580b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7882024-12-14T04:45:35.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2145 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5c0c8168-b73f-4d1d-96d9-d9105b58580b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7892024-12-14T04:45:36.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2929c500-1a1b-4058-85d4-1d9d6ac01a22 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7902024-12-14T04:45:36.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2929c500-1a1b-4058-85d4-1d9d6ac01a22 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7912024-12-14T04:45:36.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1953 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2929c500-1a1b-4058-85d4-1d9d6ac01a22 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
7922024-12-14T04:45:36.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 5857f80d-a691-47ec-af33-67c53556c17f roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
7932024-12-14T04:45:36.626ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 5857f80d-a691-47ec-af33-67c53556c17f resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
7942024-12-14T04:45:36.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3107 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 5857f80d-a691-47ec-af33-67c53556c17f response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
7952024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
7962024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
7972024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7982024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7992024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
8002024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8012024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
8022024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
8032024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8042024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
8052024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
8062024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8072024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
8082024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8092024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8102024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
8112024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8122024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
8132024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8142024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
8152024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8162024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
8172024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
8182024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8192024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
8202024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8212024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
8222024-12-14T04:45:36.882ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8232024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
8242024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8252024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8262024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8272024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8282024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8292024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8302024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8312024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8322024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8332024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8342024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8352024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8362024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8372024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8382024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8392024-12-14T04:45:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8402024-12-14T04:45:36.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
8412024-12-14T04:45:36.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
8422024-12-14T04:45:36.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8432024-12-14T04:45:36.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8442024-12-14T04:45:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8452024-12-14T04:45:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8462024-12-14T04:45:36.885ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1310, tv_nsec: 982227565 } })
8472024-12-14T04:45:36.885ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
8482024-12-14T04:45:36.885ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.647525ms iteration = 22 status = {"errors":[],"records_set_to_done":[]}
8492024-12-14T04:45:36.886ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.84138ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8502024-12-14T04:45:36.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1310, tv_nsec: 983512939 } })
8512024-12-14T04:45:36.886ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
8522024-12-14T04:45:36.888ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 5.76668ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
8532024-12-14T04:45:36.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542024-12-14T04:45:36.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8552024-12-14T04:45:36.889ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8562024-12-14T04:45:36.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8572024-12-14T04:45:36.890ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8582024-12-14T04:45:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8592024-12-14T04:45:36.891ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8602024-12-14T04:45:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8612024-12-14T04:45:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8622024-12-14T04:45:36.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8632024-12-14T04:45:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8642024-12-14T04:45:36.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8652024-12-14T04:45:36.893ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8662024-12-14T04:45:36.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8672024-12-14T04:45:36.893ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8682024-12-14T04:45:36.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8692024-12-14T04:45:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8702024-12-14T04:45:36.895ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8712024-12-14T04:45:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8722024-12-14T04:45:36.895ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8732024-12-14T04:45:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8742024-12-14T04:45:36.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8752024-12-14T04:45:36.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8762024-12-14T04:45:36.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8772024-12-14T04:45:36.902ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8782024-12-14T04:45:36.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8792024-12-14T04:45:36.904ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 21.986012ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8802024-12-14T04:45:36.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8812024-12-14T04:45:36.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8822024-12-14T04:45:36.909ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8832024-12-14T04:45:36.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8842024-12-14T04:45:36.911ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1311, tv_nsec: 8620913 } })
8852024-12-14T04:45:36.912ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
8862024-12-14T04:45:36.924ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8872024-12-14T04:45:36.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8882024-12-14T04:45:36.927ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8892024-12-14T04:45:36.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8902024-12-14T04:45:36.931ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8912024-12-14T04:45:36.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8922024-12-14T04:45:36.932ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8932024-12-14T04:45:36.932ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
8942024-12-14T04:45:36.932ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8952024-12-14T04:45:36.932ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8962024-12-14T04:45:36.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8972024-12-14T04:45:36.934ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8982024-12-14T04:45:36.934ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8992024-12-14T04:45:36.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9002024-12-14T04:45:36.935ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9012024-12-14T04:45:36.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9022024-12-14T04:45:36.938ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9032024-12-14T04:45:36.938ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042024-12-14T04:45:36.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9052024-12-14T04:45:36.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9062024-12-14T04:45:36.940ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9072024-12-14T04:45:36.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9082024-12-14T04:45:36.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9092024-12-14T04:45:36.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9102024-12-14T04:45:36.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
9112024-12-14T04:45:36.943ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9122024-12-14T04:45:36.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9132024-12-14T04:45:36.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
9142024-12-14T04:45:36.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9152024-12-14T04:45:36.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9162024-12-14T04:45:36.945ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 63.367081ms iteration = 22 status = 0
9172024-12-14T04:45:36.946ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9182024-12-14T04:45:36.946ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 62.819767ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
9192024-12-14T04:45:36.961ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
9202024-12-14T04:45:36.961ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 78.075954ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
9212024-12-14T04:45:36.962ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9222024-12-14T04:45:36.962ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9232024-12-14T04:45:36.962ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae0efc6a-5014-4a7d-ae77-e19d9b8c07ce", "content-length": "26", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9242024-12-14T04:45:36.962ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9252024-12-14T04:45:36.964ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9262024-12-14T04:45:36.967ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.796275ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9272024-12-14T04:45:36.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9282024-12-14T04:45:36.974ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9292024-12-14T04:45:36.974ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9302024-12-14T04:45:36.974ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3536237-49da-44fb-b240-0c09a759ad06", "content-length": "26", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9312024-12-14T04:45:36.974ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9322024-12-14T04:45:36.975ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 93.64175ms iteration = 22 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
9332024-12-14T04:45:36.975ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9342024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9352024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9362024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9372024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9382024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9392024-12-14T04:45:36.976ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9402024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9412024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9422024-12-14T04:45:36.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9432024-12-14T04:45:37.004ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
9442024-12-14T04:45:37.004ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9452024-12-14T04:45:37.004ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11cda20b-317c-4ac2-a853-b6673c2f7354", "content-length": "26", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9462024-12-14T04:45:37.004ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9472024-12-14T04:45:37.006ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
9482024-12-14T04:45:37.006ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 124.648948ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
9492024-12-14T04:45:37.007ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9502024-12-14T04:45:37.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9512024-12-14T04:45:37.009ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9522024-12-14T04:45:37.011ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9532024-12-14T04:45:37.011ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9542024-12-14T04:45:37.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9552024-12-14T04:45:37.013ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9562024-12-14T04:45:37.016ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9572024-12-14T04:45:37.016ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
9582024-12-14T04:45:37.016ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 134.646118ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
9592024-12-14T04:45:37.018ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9602024-12-14T04:45:37.018ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9612024-12-14T04:45:37.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9622024-12-14T04:45:37.029ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9632024-12-14T04:45:37.029ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad271a6a-ca8a-4219-b2db-96c4c151bd02", "content-length": "1", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9642024-12-14T04:45:37.031ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9652024-12-14T04:45:37.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9662024-12-14T04:45:37.033ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 151.422456ms iteration = 22 status = null
9672024-12-14T04:45:37.044ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 162.603107ms iteration = 22 status = {}
9682024-12-14T04:45:37.055ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 173.592413ms iteration = 26 status = {"physical_disks_added":0}
9692024-12-14T04:45:37.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1545a128-e207-47d2-b4e2-53549b3f4706 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
9702024-12-14T04:45:37.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1545a128-e207-47d2-b4e2-53549b3f4706 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
9712024-12-14T04:45:37.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2254 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1545a128-e207-47d2-b4e2-53549b3f4706 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
9722024-12-14T04:45:37.068ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9732024-12-14T04:45:37.070ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1110de94-36b4-4a69-b08b-5def1b76c0cc", "content-length": "691", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9742024-12-14T04:45:37.070ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
9752024-12-14T04:45:37.070ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
9762024-12-14T04:45:37.070ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
9772024-12-14T04:45:37.071ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 189.739084ms iteration = 22 status = {}
9782024-12-14T04:45:37.088ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
9792024-12-14T04:45:37.088ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9802024-12-14T04:45:37.089ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a860bdea-bb2f-45f0-a03e-661ea453a774", "content-length": "372", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9812024-12-14T04:45:37.089ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9822024-12-14T04:45:37.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9832024-12-14T04:45:37.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9842024-12-14T04:45:37.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9852024-12-14T04:45:37.125ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
9862024-12-14T04:45:37.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9872024-12-14T04:45:37.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9882024-12-14T04:45:37.130ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
9892024-12-14T04:45:37.130ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
9902024-12-14T04:45:37.130ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "630c6695-1c3e-44e1-9cb3-5cccb8b3e699", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9912024-12-14T04:45:37.130ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9922024-12-14T04:45:37.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9932024-12-14T04:45:37.133ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9942024-12-14T04:45:37.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9952024-12-14T04:45:37.135ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9962024-12-14T04:45:37.135ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "9457eafd-e5d7-41bf-9b17-7f19732bd2fe", "content-length": "134", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
9972024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
9982024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
9992024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
10002024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
10012024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10022024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10032024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10042024-12-14T04:45:37.135ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10052024-12-14T04:45:37.136ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10062024-12-14T04:45:37.136ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10072024-12-14T04:45:37.136ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10082024-12-14T04:45:37.136ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
10092024-12-14T04:45:37.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10102024-12-14T04:45:37.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10112024-12-14T04:45:37.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10122024-12-14T04:45:37.138ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
10132024-12-14T04:45:37.138ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "70bde941-243d-4c84-ac44-6a2167c4f1e4", "content-length": "2", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
10142024-12-14T04:45:37.138ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
10152024-12-14T04:45:37.139ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0782fbdf-a519-443c-af02-df1151ac2a4a", "content-length": "52", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
10162024-12-14T04:45:37.139ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
10172024-12-14T04:45:37.139ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
10182024-12-14T04:45:37.139ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
10192024-12-14T04:45:37.139ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10202024-12-14T04:45:37.231ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a00cba53-dba1-47cb-809d-4ec3b5e70fe1", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
10212024-12-14T04:45:37.231ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
10222024-12-14T04:45:37.231ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
10232024-12-14T04:45:37.231ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "df849d21-96c0-4ab5-aaa2-6444f28ed06c", "date": "Sat, 14 Dec 2024 04:45:36 GMT"} })
10242024-12-14T04:45:37.233ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 350.733669ms iteration = 23 status = {}
10252024-12-14T04:45:37.233ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c55a2464-04e1-42c1-97a0-3cbcb195054f", "content-length": "682", "date": "Sat, 14 Dec 2024 04:45:37 GMT"} })
10262024-12-14T04:45:37.233ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10272024-12-14T04:45:37.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10282024-12-14T04:45:37.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10292024-12-14T04:45:37.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10302024-12-14T04:45:37.240ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10312024-12-14T04:45:37.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10322024-12-14T04:45:37.242ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10332024-12-14T04:45:37.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10342024-12-14T04:45:37.248ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 366.754586ms iteration = 22 status = {}
10352024-12-14T04:45:37.710ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:63351
10362024-12-14T04:45:37.710ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:63351 req_id = 336c2f41-07f9-4c6d-b199-a4e70fded503 roles = RoleSet { roles: {} } uri = /metrics/producers
10372024-12-14T04:45:37.710ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:63351 req_id = 336c2f41-07f9-4c6d-b199-a4e70fded503 resource = Database result = Ok(()) uri = /metrics/producers
10382024-12-14T04:45:37.718ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 7c8e60d1-cbc0-4976-b68e-162aaf207501
10392024-12-14T04:45:37.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8099 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:63351 req_id = 336c2f41-07f9-4c6d-b199-a4e70fded503 response_code = 201 uri = /metrics/producers
10402024-12-14T04:45:38.084ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af2b08ab-a515-4625-b0c2-fe50c3d94d10 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10412024-12-14T04:45:38.084ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af2b08ab-a515-4625-b0c2-fe50c3d94d10 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10422024-12-14T04:45:38.087ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2679 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af2b08ab-a515-4625-b0c2-fe50c3d94d10 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10432024-12-14T04:45:39.107ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f4895261-c594-4986-823a-d8b3b90d824d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10442024-12-14T04:45:39.107ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f4895261-c594-4986-823a-d8b3b90d824d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10452024-12-14T04:45:39.109ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2538 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f4895261-c594-4986-823a-d8b3b90d824d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10462024-12-14T04:45:40.129ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 746333dd-80f2-49d0-a9a4-937ee0cb12d0 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10472024-12-14T04:45:40.129ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 746333dd-80f2-49d0-a9a4-937ee0cb12d0 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10482024-12-14T04:45:40.131ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 746333dd-80f2-49d0-a9a4-937ee0cb12d0 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10492024-12-14T04:45:41.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e2461c2-5f3f-41d3-a7ba-10862bf22d74 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10502024-12-14T04:45:41.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e2461c2-5f3f-41d3-a7ba-10862bf22d74 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10512024-12-14T04:45:41.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1977 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e2461c2-5f3f-41d3-a7ba-10862bf22d74 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10522024-12-14T04:45:42.173ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7b201b18-f470-4935-8856-5d506edfbbbe roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10532024-12-14T04:45:42.173ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7b201b18-f470-4935-8856-5d506edfbbbe resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10542024-12-14T04:45:42.174ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1951 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7b201b18-f470-4935-8856-5d506edfbbbe response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10552024-12-14T04:45:42.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1413 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 97985578-c61d-463b-a866-d359fda9718c response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
10562024-12-14T04:45:43.194ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3df30d64-c2e3-4bf5-a5f3-645d37a53625 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10572024-12-14T04:45:43.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3df30d64-c2e3-4bf5-a5f3-645d37a53625 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10582024-12-14T04:45:43.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2475 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3df30d64-c2e3-4bf5-a5f3-645d37a53625 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10592024-12-14T04:45:44.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1a6c2a0-67f7-43f0-859a-0ab042ea7bb6 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10602024-12-14T04:45:44.216ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1a6c2a0-67f7-43f0-859a-0ab042ea7bb6 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10612024-12-14T04:45:44.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1939 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1a6c2a0-67f7-43f0-859a-0ab042ea7bb6 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10622024-12-14T04:45:45.237ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f5a8023e-1a8d-4e2b-95f6-e5fc573e5bf9 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10632024-12-14T04:45:45.237ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f5a8023e-1a8d-4e2b-95f6-e5fc573e5bf9 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10642024-12-14T04:45:45.239ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2178 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f5a8023e-1a8d-4e2b-95f6-e5fc573e5bf9 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10652024-12-14T04:45:46.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9401f1fd-a325-428a-9abe-8b16ecee9627 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10662024-12-14T04:45:46.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9401f1fd-a325-428a-9abe-8b16ecee9627 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10672024-12-14T04:45:46.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2237 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9401f1fd-a325-428a-9abe-8b16ecee9627 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10682024-12-14T04:45:46.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
10692024-12-14T04:45:46.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10702024-12-14T04:45:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10712024-12-14T04:45:46.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
10722024-12-14T04:45:46.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10732024-12-14T04:45:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10742024-12-14T04:45:46.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10752024-12-14T04:45:46.912ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10762024-12-14T04:45:46.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10772024-12-14T04:45:46.914ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.60757ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10782024-12-14T04:45:46.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10792024-12-14T04:45:46.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10802024-12-14T04:45:46.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10812024-12-14T04:45:46.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10822024-12-14T04:45:46.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10832024-12-14T04:45:46.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10842024-12-14T04:45:46.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
10852024-12-14T04:45:46.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10862024-12-14T04:45:46.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10872024-12-14T04:45:46.929ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
10882024-12-14T04:45:46.929ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.995964ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
10892024-12-14T04:45:47.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ba6f4d4e-073f-4a7f-8051-33a4ea427feb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10902024-12-14T04:45:47.281ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ba6f4d4e-073f-4a7f-8051-33a4ea427feb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10912024-12-14T04:45:47.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2236 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ba6f4d4e-073f-4a7f-8051-33a4ea427feb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
10922024-12-14T04:45:47.770ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:41967
10932024-12-14T04:45:47.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 1d0535eb-4540-48c2-8604-10d1e7cf5c0e roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
10942024-12-14T04:45:47.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 1d0535eb-4540-48c2-8604-10d1e7cf5c0e resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
10952024-12-14T04:45:47.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2549 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 1d0535eb-4540-48c2-8604-10d1e7cf5c0e response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
10962024-12-14T04:45:47.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 0c877203-c9ee-41d9-9098-98af332d797f roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
10972024-12-14T04:45:47.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 0c877203-c9ee-41d9-9098-98af332d797f resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
10982024-12-14T04:45:47.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1836 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 0c877203-c9ee-41d9-9098-98af332d797f response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
10992024-12-14T04:45:48.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bcd6158b-1346-41e5-b0bc-a2f4f7e94440 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11002024-12-14T04:45:48.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bcd6158b-1346-41e5-b0bc-a2f4f7e94440 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11012024-12-14T04:45:48.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1830 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bcd6158b-1346-41e5-b0bc-a2f4f7e94440 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11022024-12-14T04:45:49.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 55fdad29-101c-474d-9485-6bb9466d4d00 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11032024-12-14T04:45:49.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 55fdad29-101c-474d-9485-6bb9466d4d00 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11042024-12-14T04:45:49.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2074 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 55fdad29-101c-474d-9485-6bb9466d4d00 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11052024-12-14T04:45:50.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 73adbc5f-9d7a-4ca1-b870-1f0080057b85 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11062024-12-14T04:45:50.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 73adbc5f-9d7a-4ca1-b870-1f0080057b85 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11072024-12-14T04:45:50.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 73adbc5f-9d7a-4ca1-b870-1f0080057b85 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11082024-12-14T04:45:51.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 449e8973-541e-4255-abfa-8823210f5d72 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11092024-12-14T04:45:51.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 449e8973-541e-4255-abfa-8823210f5d72 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11102024-12-14T04:45:51.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1937 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 449e8973-541e-4255-abfa-8823210f5d72 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11112024-12-14T04:45:52.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 32fc4ba9-e965-4f03-83df-520a2f155930 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11122024-12-14T04:45:52.389ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 32fc4ba9-e965-4f03-83df-520a2f155930 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11132024-12-14T04:45:52.391ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2371 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 32fc4ba9-e965-4f03-83df-520a2f155930 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11142024-12-14T04:45:52.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1406 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 13854118-77da-421c-9069-a833f204d2ca response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
11152024-12-14T04:45:53.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c44d869-8c5d-49aa-bfc3-65024d08dc30 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11162024-12-14T04:45:53.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c44d869-8c5d-49aa-bfc3-65024d08dc30 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11172024-12-14T04:45:53.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1825 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c44d869-8c5d-49aa-bfc3-65024d08dc30 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11182024-12-14T04:45:54.080ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:63179
11192024-12-14T04:45:54.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:63179 req_id = 09d87a4a-0dd3-42f5-9287-7bf1a9ff8521 roles = RoleSet { roles: {} } uri = /metrics/producers
11202024-12-14T04:45:54.081ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:63179 req_id = 09d87a4a-0dd3-42f5-9287-7bf1a9ff8521 resource = Database result = Ok(()) uri = /metrics/producers
11212024-12-14T04:45:54.089ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 4f6cecdf-822e-4790-965b-a20901bfe59f
11222024-12-14T04:45:54.089ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8408 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:63179 req_id = 09d87a4a-0dd3-42f5-9287-7bf1a9ff8521 response_code = 201 uri = /metrics/producers
11232024-12-14T04:45:54.432ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 978203d4-dde3-4514-8fd4-02421cc5a1b8 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11242024-12-14T04:45:54.432ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 978203d4-dde3-4514-8fd4-02421cc5a1b8 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11252024-12-14T04:45:54.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2113 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 978203d4-dde3-4514-8fd4-02421cc5a1b8 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11262024-12-14T04:45:55.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = de513724-81e2-4cb9-8ce8-3ce0ed834035 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11272024-12-14T04:45:55.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = de513724-81e2-4cb9-8ce8-3ce0ed834035 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11282024-12-14T04:45:55.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = de513724-81e2-4cb9-8ce8-3ce0ed834035 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11292024-12-14T04:45:56.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f073de83-9fe4-429d-9a72-be0bfe0167d3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11302024-12-14T04:45:56.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f073de83-9fe4-429d-9a72-be0bfe0167d3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11312024-12-14T04:45:56.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2016 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f073de83-9fe4-429d-9a72-be0bfe0167d3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11322024-12-14T04:45:56.531ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:44924
11332024-12-14T04:45:56.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44924 req_id = d30bc9df-145a-4761-83ad-b963a3c97823 roles = RoleSet { roles: {} } uri = /metrics/producers
11342024-12-14T04:45:56.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44924 req_id = d30bc9df-145a-4761-83ad-b963a3c97823 resource = Database result = Ok(()) uri = /metrics/producers
11352024-12-14T04:45:56.543ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11362024-12-14T04:45:56.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12086 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44924 req_id = d30bc9df-145a-4761-83ad-b963a3c97823 response_code = 201 uri = /metrics/producers
11372024-12-14T04:45:56.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
11382024-12-14T04:45:56.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11392024-12-14T04:45:56.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11402024-12-14T04:45:56.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
11412024-12-14T04:45:56.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11422024-12-14T04:45:56.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11432024-12-14T04:45:56.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11442024-12-14T04:45:56.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11452024-12-14T04:45:56.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11462024-12-14T04:45:56.885ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.882142ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11472024-12-14T04:45:56.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11482024-12-14T04:45:56.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11492024-12-14T04:45:56.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11502024-12-14T04:45:56.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11512024-12-14T04:45:56.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11522024-12-14T04:45:56.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11532024-12-14T04:45:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
11542024-12-14T04:45:56.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11552024-12-14T04:45:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11562024-12-14T04:45:56.901ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
11572024-12-14T04:45:56.901ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.726207ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
11582024-12-14T04:45:57.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3b5413dc-1780-4fbe-95f7-349c66fe427c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11592024-12-14T04:45:57.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3b5413dc-1780-4fbe-95f7-349c66fe427c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11602024-12-14T04:45:57.498ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1923 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3b5413dc-1780-4fbe-95f7-349c66fe427c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11612024-12-14T04:45:58.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b9235738-ce79-4675-bcd2-8338cf921a69 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11622024-12-14T04:45:58.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b9235738-ce79-4675-bcd2-8338cf921a69 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11632024-12-14T04:45:58.519ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1902 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b9235738-ce79-4675-bcd2-8338cf921a69 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11642024-12-14T04:45:59.540ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e56bee5-76ef-4c38-9c44-96575bc7a5c4 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11652024-12-14T04:45:59.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e56bee5-76ef-4c38-9c44-96575bc7a5c4 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11662024-12-14T04:45:59.542ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2347 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e56bee5-76ef-4c38-9c44-96575bc7a5c4 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11672024-12-14T04:46:00.562ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c8754cc4-54e9-4a4f-82c0-0b695f7efd82 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11682024-12-14T04:46:00.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c8754cc4-54e9-4a4f-82c0-0b695f7efd82 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11692024-12-14T04:46:00.564ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2325 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c8754cc4-54e9-4a4f-82c0-0b695f7efd82 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11702024-12-14T04:46:01.584ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b0d89695-54eb-4d01-8523-e4a921bb9f1a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11712024-12-14T04:46:01.584ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b0d89695-54eb-4d01-8523-e4a921bb9f1a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11722024-12-14T04:46:01.585ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1621 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b0d89695-54eb-4d01-8523-e4a921bb9f1a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11732024-12-14T04:46:02.605ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 26517b00-7b8d-4c1e-b58a-c28a87013d69 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11742024-12-14T04:46:02.605ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 26517b00-7b8d-4c1e-b58a-c28a87013d69 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11752024-12-14T04:46:02.607ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2215 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 26517b00-7b8d-4c1e-b58a-c28a87013d69 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11762024-12-14T04:46:02.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1223 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = cd8ebf57-4fca-4109-a2f6-751846ba01d9 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
11772024-12-14T04:46:03.627ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = aa0cee74-7af8-4fa7-8ceb-19198a4179dc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11782024-12-14T04:46:03.628ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = aa0cee74-7af8-4fa7-8ceb-19198a4179dc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11792024-12-14T04:46:03.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2533 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = aa0cee74-7af8-4fa7-8ceb-19198a4179dc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11802024-12-14T04:46:04.650ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b62a3013-14d1-4f7a-8e32-04d04c084ae6 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11812024-12-14T04:46:04.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b62a3013-14d1-4f7a-8e32-04d04c084ae6 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11822024-12-14T04:46:04.653ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2440 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b62a3013-14d1-4f7a-8e32-04d04c084ae6 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11832024-12-14T04:46:05.672ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a5c6b4a9-4aaa-45f5-a6ce-711db7a68d41 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11842024-12-14T04:46:05.672ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a5c6b4a9-4aaa-45f5-a6ce-711db7a68d41 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11852024-12-14T04:46:05.674ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2213 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a5c6b4a9-4aaa-45f5-a6ce-711db7a68d41 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11862024-12-14T04:46:06.658ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = f7102fc3-2590-494a-abbe-87cbff7a2458 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
11872024-12-14T04:46:06.658ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = f7102fc3-2590-494a-abbe-87cbff7a2458 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
11882024-12-14T04:46:06.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4006 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = f7102fc3-2590-494a-abbe-87cbff7a2458 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
11892024-12-14T04:46:06.693ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c257fa7-f51e-4217-b5bf-d2c227e5e9fe roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11902024-12-14T04:46:06.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c257fa7-f51e-4217-b5bf-d2c227e5e9fe resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11912024-12-14T04:46:06.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1870 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c257fa7-f51e-4217-b5bf-d2c227e5e9fe response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
11922024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
11932024-12-14T04:46:06.881ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
11942024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
11952024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
11962024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
11972024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11982024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11992024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12002024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
12012024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12022024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
12032024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12042024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
12052024-12-14T04:46:06.881ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 04:36:06.881325612 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
12062024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
12072024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
12082024-12-14T04:46:06.881ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
12092024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
12102024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
12112024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12122024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
12132024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12142024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
12152024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
12162024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
12172024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
12182024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
12192024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
12202024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12212024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
12222024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
12232024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
12242024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12252024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
12262024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
12272024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
12282024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
12292024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12302024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
12312024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
12322024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12332024-12-14T04:46:06.881ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832
12342024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
12352024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.06\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832"}
12362024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
12372024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
12382024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
12392024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12402024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12412024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
12422024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
12432024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
12442024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12452024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
12462024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12472024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12482024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12492024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12502024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12512024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
12522024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
12532024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12542024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
12552024-12-14T04:46:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
12562024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12572024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
12582024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12592024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12602024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12612024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12622024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12632024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12642024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
12652024-12-14T04:46:06.882ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ba1572d-df24-4bd2-be03-9a99edc867d8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1340, tv_nsec: 979503294 } })
12662024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12672024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12682024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
12692024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
12702024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
12712024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
12722024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12732024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
12742024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
12752024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12762024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
12772024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12782024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12792024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
12802024-12-14T04:46:06.882ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.013516ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
12812024-12-14T04:46:06.882ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2224041a-a9fc-46d2-b9ed-ef98dc01f696.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("229c9d57-e318-46f6-8df9-9f602a126f12.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1340, tv_nsec: 980037718 } })
12822024-12-14T04:46:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
12832024-12-14T04:46:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 907.994\u{b5}s iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
12842024-12-14T04:46:06.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12852024-12-14T04:46:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12862024-12-14T04:46:06.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12872024-12-14T04:46:06.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12882024-12-14T04:46:06.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12892024-12-14T04:46:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12902024-12-14T04:46:06.887ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1340, tv_nsec: 984930246 } })
12912024-12-14T04:46:06.887ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12922024-12-14T04:46:06.887ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
12932024-12-14T04:46:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12942024-12-14T04:46:06.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12952024-12-14T04:46:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12962024-12-14T04:46:06.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12972024-12-14T04:46:06.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12982024-12-14T04:46:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12992024-12-14T04:46:06.891ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.185075ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
13002024-12-14T04:46:06.892ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 04:36:06.881325612 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
13012024-12-14T04:46:06.892ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.937585ms iteration = 12 status = {"expiration":"2024-12-14T04:36:06.881325612Z","pruned":[]}
13022024-12-14T04:46:06.892ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13032024-12-14T04:46:06.892ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.295414ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
13042024-12-14T04:46:06.893ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13052024-12-14T04:46:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13062024-12-14T04:46:06.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13072024-12-14T04:46:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13082024-12-14T04:46:06.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
13092024-12-14T04:46:06.894ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
13102024-12-14T04:46:06.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
13112024-12-14T04:46:06.895ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13122024-12-14T04:46:06.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13132024-12-14T04:46:06.896ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1340, tv_nsec: 993674454 } })
13142024-12-14T04:46:06.896ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
13152024-12-14T04:46:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13162024-12-14T04:46:06.896ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13172024-12-14T04:46:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13182024-12-14T04:46:06.901ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13192024-12-14T04:46:06.902ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13202024-12-14T04:46:06.904ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1341, tv_nsec: 1391124 } })
13212024-12-14T04:46:06.904ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
13222024-12-14T04:46:06.904ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13232024-12-14T04:46:06.904ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13242024-12-14T04:46:06.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13252024-12-14T04:46:06.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13262024-12-14T04:46:06.906ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13272024-12-14T04:46:06.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13282024-12-14T04:46:06.909ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13292024-12-14T04:46:06.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13302024-12-14T04:46:06.938ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13312024-12-14T04:46:06.938ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
13322024-12-14T04:46:06.938ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13332024-12-14T04:46:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13342024-12-14T04:46:06.943ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13352024-12-14T04:46:06.943ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 61.742378ms iteration = 23 status = {"errors":[],"records_set_to_done":[]}
13362024-12-14T04:46:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13372024-12-14T04:46:06.943ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13382024-12-14T04:46:06.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
13392024-12-14T04:46:06.945ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13402024-12-14T04:46:06.945ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13412024-12-14T04:46:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13422024-12-14T04:46:06.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13432024-12-14T04:46:06.949ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13442024-12-14T04:46:06.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13452024-12-14T04:46:06.950ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 68.663439ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
13462024-12-14T04:46:06.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13472024-12-14T04:46:06.951ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13482024-12-14T04:46:06.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13492024-12-14T04:46:06.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13502024-12-14T04:46:06.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13512024-12-14T04:46:06.952ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
13522024-12-14T04:46:06.952ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
13532024-12-14T04:46:06.952ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 71.344419ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
13542024-12-14T04:46:06.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13552024-12-14T04:46:06.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13562024-12-14T04:46:06.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13572024-12-14T04:46:06.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13582024-12-14T04:46:06.958ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13592024-12-14T04:46:06.958ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13602024-12-14T04:46:06.958ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13612024-12-14T04:46:06.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13622024-12-14T04:46:06.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13632024-12-14T04:46:06.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13642024-12-14T04:46:06.960ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13652024-12-14T04:46:06.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13662024-12-14T04:46:06.961ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13672024-12-14T04:46:06.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13682024-12-14T04:46:06.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13692024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13702024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
13712024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13722024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13732024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13742024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13752024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13762024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13772024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
13782024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
13792024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
13802024-12-14T04:46:06.963ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13812024-12-14T04:46:06.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13822024-12-14T04:46:06.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13832024-12-14T04:46:06.965ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
13842024-12-14T04:46:06.965ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
13852024-12-14T04:46:06.965ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 83.729381ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
13862024-12-14T04:46:06.965ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13872024-12-14T04:46:06.965ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.022059ms iteration = 23 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
13882024-12-14T04:46:06.965ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
13892024-12-14T04:46:06.965ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 83.442095ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
13902024-12-14T04:46:06.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13912024-12-14T04:46:06.966ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13922024-12-14T04:46:06.966ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13932024-12-14T04:46:06.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13942024-12-14T04:46:06.968ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13952024-12-14T04:46:06.968ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13962024-12-14T04:46:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
13972024-12-14T04:46:06.968ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13982024-12-14T04:46:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13992024-12-14T04:46:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14002024-12-14T04:46:06.970ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14012024-12-14T04:46:06.973ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
14022024-12-14T04:46:06.973ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14032024-12-14T04:46:06.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14042024-12-14T04:46:06.975ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 94.152523ms iteration = 23 status = 0
14052024-12-14T04:46:06.977ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
14062024-12-14T04:46:06.977ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 95.721824ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
14072024-12-14T04:46:06.984ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 103.21964ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
14082024-12-14T04:46:06.995ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14092024-12-14T04:46:06.995ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
14102024-12-14T04:46:06.995ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14112024-12-14T04:46:06.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14122024-12-14T04:46:06.996ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.397956ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
14132024-12-14T04:46:06.997ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
14142024-12-14T04:46:06.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
14152024-12-14T04:46:06.999ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
14162024-12-14T04:46:07.002ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 121.233048ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14172024-12-14T04:46:07.005ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
14182024-12-14T04:46:07.005ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
14192024-12-14T04:46:07.005ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
14202024-12-14T04:46:07.005ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14212024-12-14T04:46:07.005ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14222024-12-14T04:46:07.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14232024-12-14T04:46:07.007ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14242024-12-14T04:46:07.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14252024-12-14T04:46:07.007ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14262024-12-14T04:46:07.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14272024-12-14T04:46:07.008ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14282024-12-14T04:46:07.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14292024-12-14T04:46:07.010ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14302024-12-14T04:46:07.011ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14312024-12-14T04:46:07.011ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14322024-12-14T04:46:07.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14332024-12-14T04:46:07.011ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14342024-12-14T04:46:07.011ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14352024-12-14T04:46:07.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14362024-12-14T04:46:07.015ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14372024-12-14T04:46:07.017ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 135.580053ms iteration = 27 status = {"physical_disks_added":0}
14382024-12-14T04:46:07.018ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14392024-12-14T04:46:07.018ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14402024-12-14T04:46:07.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14412024-12-14T04:46:07.020ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
14422024-12-14T04:46:07.020ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14432024-12-14T04:46:07.020ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14442024-12-14T04:46:07.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14452024-12-14T04:46:07.022ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14462024-12-14T04:46:07.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14472024-12-14T04:46:07.025ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14482024-12-14T04:46:07.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14492024-12-14T04:46:07.027ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
14502024-12-14T04:46:07.027ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
14512024-12-14T04:46:07.027ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 04:35:18.025163 UTC dns_group = internal generation = 1
14522024-12-14T04:46:07.027ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 146.59737ms iteration = 13 status = {"generation":1}
14532024-12-14T04:46:07.038ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14542024-12-14T04:46:07.038ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
14552024-12-14T04:46:07.038ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 156.95006ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
14562024-12-14T04:46:07.040ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
14572024-12-14T04:46:07.040ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 0 version = 2
14582024-12-14T04:46:07.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14592024-12-14T04:46:07.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14602024-12-14T04:46:07.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14612024-12-14T04:46:07.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14622024-12-14T04:46:07.045ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14632024-12-14T04:46:07.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14642024-12-14T04:46:07.049ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T04:35:03.711237Z, time_modified: 2024-12-14T04:35:03.711237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }) }
14652024-12-14T04:46:07.049ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 167.797303ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]}},"default_endpoint":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
14662024-12-14T04:46:07.052ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 1 version = 2
14672024-12-14T04:46:07.052ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
14682024-12-14T04:46:07.053ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 04:35:19.721992 UTC dns_group = external generation = 2
14692024-12-14T04:46:07.053ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 171.483228ms iteration = 13 status = {"generation":2}
14702024-12-14T04:46:07.064ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
14712024-12-14T04:46:07.064ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 182.193527ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
14722024-12-14T04:46:07.148ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
14732024-12-14T04:46:07.178ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
14742024-12-14T04:46:07.178ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14752024-12-14T04:46:07.179ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e098aaf-8945-40f4-9d46-12c8f3b65334", "content-length": "26", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} })
14762024-12-14T04:46:07.179ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14772024-12-14T04:46:07.181ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14782024-12-14T04:46:07.199ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
14792024-12-14T04:46:07.199ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e4546532-5bb2-4eb1-acbc-352b171fdbed", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14802024-12-14T04:46:07.199ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14812024-12-14T04:46:07.201ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f9acd900-da7f-410b-abea-fca2a0c96c72", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14822024-12-14T04:46:07.201ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14832024-12-14T04:46:07.202ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14842024-12-14T04:46:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 321.172279ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
14852024-12-14T04:46:07.217ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
14862024-12-14T04:46:07.217ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14872024-12-14T04:46:07.217ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d58f95e-9c21-46bd-bab3-fb94e610ff64", "content-length": "26", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
14882024-12-14T04:46:07.217ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14892024-12-14T04:46:07.218ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
14902024-12-14T04:46:07.218ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14912024-12-14T04:46:07.218ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "40fc75a0-eb64-45ba-8369-febaac5dacd5", "content-length": "372", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} })
14922024-12-14T04:46:07.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14932024-12-14T04:46:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14942024-12-14T04:46:07.219ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
14952024-12-14T04:46:07.231ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
14962024-12-14T04:46:07.232ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "373b276a-c1ad-46dd-b3e3-3b5b5748b3fd", "content-length": "691", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} })
14972024-12-14T04:46:07.232ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
14982024-12-14T04:46:07.232ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
14992024-12-14T04:46:07.232ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
15002024-12-14T04:46:07.232ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
15012024-12-14T04:46:07.232ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
15022024-12-14T04:46:07.232ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a3d905ee-42c2-489e-bd4d-0d092606d4be", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} })
15032024-12-14T04:46:07.232ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15042024-12-14T04:46:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15052024-12-14T04:46:07.233ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 352.31287ms iteration = 23 status = {}
15062024-12-14T04:46:07.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15072024-12-14T04:46:07.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15082024-12-14T04:46:07.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15092024-12-14T04:46:07.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15102024-12-14T04:46:07.240ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
15112024-12-14T04:46:07.240ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15122024-12-14T04:46:07.244ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "80a400f9-3718-477a-ad02-9619e5e8eb78", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} })
15132024-12-14T04:46:07.245ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 364.403895ms iteration = 24 status = {}
15142024-12-14T04:46:07.250ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
15152024-12-14T04:46:07.250ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15162024-12-14T04:46:07.250ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c281c78-f50e-4b71-8c1b-f57dca591b94", "content-length": "26", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15172024-12-14T04:46:07.250ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15182024-12-14T04:46:07.251ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15192024-12-14T04:46:07.313ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15202024-12-14T04:46:07.314ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4b64558-9237-45e0-a528-3d853e961366", "content-length": "1", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15212024-12-14T04:46:07.316ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15222024-12-14T04:46:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15232024-12-14T04:46:07.318ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 436.992816ms iteration = 23 status = null
15242024-12-14T04:46:07.325ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
15252024-12-14T04:46:07.411ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
15262024-12-14T04:46:07.412ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6d691a9c-e3f0-4c30-a683-9f6a7560ea06", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15272024-12-14T04:46:07.412ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15282024-12-14T04:46:07.413ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "31d230bd-4db6-4d96-bf9e-14c04212e72f", "date": "Sat, 14 Dec 2024 04:46:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15292024-12-14T04:46:07.413ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15302024-12-14T04:46:07.414ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bb0d9d2e-b3be-441d-b767-61a98bfe7cbe", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15312024-12-14T04:46:07.414ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15322024-12-14T04:46:07.416ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
15332024-12-14T04:46:07.416ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 535.151674ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
15342024-12-14T04:46:07.450ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 568.79711ms iteration = 23 status = {}
15352024-12-14T04:46:07.497ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15362024-12-14T04:46:07.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15372024-12-14T04:46:07.500ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
15382024-12-14T04:46:07.500ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15392024-12-14T04:46:07.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15402024-12-14T04:46:07.509ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15412024-12-14T04:46:07.509ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e617a54-56d9-4fcb-a5e8-18cd7cf2a93c", "content-length": "134", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15422024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
15432024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
15442024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
15452024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
15462024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15472024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15482024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15492024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15502024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15512024-12-14T04:46:07.509ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15522024-12-14T04:46:07.510ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
15532024-12-14T04:46:07.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15542024-12-14T04:46:07.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15552024-12-14T04:46:07.512ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15562024-12-14T04:46:07.512ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8b4709c-6b62-4fc6-b879-0f66a9c2f3bf", "content-length": "2", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15572024-12-14T04:46:07.512ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15582024-12-14T04:46:07.512ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e16a42e4-df75-45a4-87ba-9b12307af6a7", "content-length": "52", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15592024-12-14T04:46:07.512ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
15602024-12-14T04:46:07.512ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15612024-12-14T04:46:07.600ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "87761c34-4bf4-4e85-9273-9d3e5257b8a5", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15622024-12-14T04:46:07.600ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
15632024-12-14T04:46:07.600ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15642024-12-14T04:46:07.600ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ef8d650-a301-4bc0-b35d-f33ab67a839a", "content-length": "682", "date": "Sat, 14 Dec 2024 04:46:07 GMT"} })
15652024-12-14T04:46:07.600ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15662024-12-14T04:46:07.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15672024-12-14T04:46:07.604ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15682024-12-14T04:46:07.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15692024-12-14T04:46:07.607ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15702024-12-14T04:46:07.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15712024-12-14T04:46:07.610ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15722024-12-14T04:46:07.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15732024-12-14T04:46:07.617ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 736.578273ms iteration = 23 status = {}
15742024-12-14T04:46:07.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5802ad87-e39e-4587-b86f-08fc5c3a43eb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15752024-12-14T04:46:07.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5802ad87-e39e-4587-b86f-08fc5c3a43eb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15762024-12-14T04:46:07.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2095 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5802ad87-e39e-4587-b86f-08fc5c3a43eb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15772024-12-14T04:46:08.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b456283-083a-4e4a-9483-d6de5fe707fb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15782024-12-14T04:46:08.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b456283-083a-4e4a-9483-d6de5fe707fb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15792024-12-14T04:46:08.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1927 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b456283-083a-4e4a-9483-d6de5fe707fb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15802024-12-14T04:46:09.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d4d26b13-8884-468e-a1aa-89bdeeed988c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15812024-12-14T04:46:09.761ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d4d26b13-8884-468e-a1aa-89bdeeed988c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15822024-12-14T04:46:09.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1777 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d4d26b13-8884-468e-a1aa-89bdeeed988c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15832024-12-14T04:46:10.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2aa738bb-5608-4b9e-ad6d-fcba1960af17 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15842024-12-14T04:46:10.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2aa738bb-5608-4b9e-ad6d-fcba1960af17 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15852024-12-14T04:46:10.784ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2186 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2aa738bb-5608-4b9e-ad6d-fcba1960af17 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15862024-12-14T04:46:11.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7002cd3b-8068-42f9-b1b7-6e0628c8dfd3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15872024-12-14T04:46:11.802ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7002cd3b-8068-42f9-b1b7-6e0628c8dfd3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15882024-12-14T04:46:11.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2132 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7002cd3b-8068-42f9-b1b7-6e0628c8dfd3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15892024-12-14T04:46:12.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1471 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 6b9c2822-1da4-4743-af6e-6704683e8dce response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
15902024-12-14T04:46:12.824ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f96d6ed9-b2c1-4276-8621-cf916cc4d15b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15912024-12-14T04:46:12.825ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f96d6ed9-b2c1-4276-8621-cf916cc4d15b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15922024-12-14T04:46:12.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1939 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f96d6ed9-b2c1-4276-8621-cf916cc4d15b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15932024-12-14T04:46:13.846ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 929c5891-da9f-40eb-9e6f-9a5985713222 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15942024-12-14T04:46:13.847ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 929c5891-da9f-40eb-9e6f-9a5985713222 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15952024-12-14T04:46:13.849ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2463 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 929c5891-da9f-40eb-9e6f-9a5985713222 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15962024-12-14T04:46:14.869ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 276e566b-4802-42ec-b08f-3347a0754093 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15972024-12-14T04:46:14.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 276e566b-4802-42ec-b08f-3347a0754093 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15982024-12-14T04:46:14.871ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2138 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 276e566b-4802-42ec-b08f-3347a0754093 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
15992024-12-14T04:46:15.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1ad9582d-416a-4ea8-9699-76d55dbe7404 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16002024-12-14T04:46:15.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1ad9582d-416a-4ea8-9699-76d55dbe7404 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16012024-12-14T04:46:15.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1ad9582d-416a-4ea8-9699-76d55dbe7404 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16022024-12-14T04:46:16.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
16032024-12-14T04:46:16.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16042024-12-14T04:46:16.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16052024-12-14T04:46:16.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
16062024-12-14T04:46:16.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16072024-12-14T04:46:16.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16082024-12-14T04:46:16.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16092024-12-14T04:46:16.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16102024-12-14T04:46:16.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16112024-12-14T04:46:16.885ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.84591ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16122024-12-14T04:46:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16132024-12-14T04:46:16.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16142024-12-14T04:46:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16152024-12-14T04:46:16.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16162024-12-14T04:46:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16172024-12-14T04:46:16.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16182024-12-14T04:46:16.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
16192024-12-14T04:46:16.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16202024-12-14T04:46:16.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16212024-12-14T04:46:16.904ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
16222024-12-14T04:46:16.904ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.779827ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
16232024-12-14T04:46:16.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37bd4765-901c-459a-bb0b-9deebc878c07 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16242024-12-14T04:46:16.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37bd4765-901c-459a-bb0b-9deebc878c07 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16252024-12-14T04:46:16.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1688 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37bd4765-901c-459a-bb0b-9deebc878c07 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16262024-12-14T04:46:17.769ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = a350134a-6142-4f6d-8ce0-cbf0810fa74e roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
16272024-12-14T04:46:17.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = a350134a-6142-4f6d-8ce0-cbf0810fa74e resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
16282024-12-14T04:46:17.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3099 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = a350134a-6142-4f6d-8ce0-cbf0810fa74e response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
16292024-12-14T04:46:17.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 90b61eeb-4ff0-4877-9afa-3cf0d0f75e21 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
16302024-12-14T04:46:17.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 90b61eeb-4ff0-4877-9afa-3cf0d0f75e21 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
16312024-12-14T04:46:17.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2441 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 90b61eeb-4ff0-4877-9afa-3cf0d0f75e21 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
16322024-12-14T04:46:17.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af5b061a-0838-4c66-8561-513770531b53 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16332024-12-14T04:46:17.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af5b061a-0838-4c66-8561-513770531b53 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16342024-12-14T04:46:17.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1990 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af5b061a-0838-4c66-8561-513770531b53 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16352024-12-14T04:46:18.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74393271-4c94-44da-9f24-1e9a60d2a967 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16362024-12-14T04:46:18.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74393271-4c94-44da-9f24-1e9a60d2a967 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16372024-12-14T04:46:18.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2244 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74393271-4c94-44da-9f24-1e9a60d2a967 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16382024-12-14T04:46:19.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f049ba87-3728-44b0-8bf6-a1f82f5ef5e5 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16392024-12-14T04:46:19.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f049ba87-3728-44b0-8bf6-a1f82f5ef5e5 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16402024-12-14T04:46:19.981ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2472 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f049ba87-3728-44b0-8bf6-a1f82f5ef5e5 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16412024-12-14T04:46:21.000ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9b5f3c9d-92ce-4caf-818c-86650c8912fd roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16422024-12-14T04:46:21.000ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9b5f3c9d-92ce-4caf-818c-86650c8912fd resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16432024-12-14T04:46:21.002ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2074 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9b5f3c9d-92ce-4caf-818c-86650c8912fd response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16442024-12-14T04:46:22.022ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f162b12f-bca5-49a8-af6f-039899993552 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16452024-12-14T04:46:22.022ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f162b12f-bca5-49a8-af6f-039899993552 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16462024-12-14T04:46:22.024ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1935 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f162b12f-bca5-49a8-af6f-039899993552 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16472024-12-14T04:46:22.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1308 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = af2a1bc5-da5d-410b-9889-9c02f5513eac response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
16482024-12-14T04:46:23.042ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 95650b45-a7ad-4cfc-8d70-79234ce5fd93 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16492024-12-14T04:46:23.043ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 95650b45-a7ad-4cfc-8d70-79234ce5fd93 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16502024-12-14T04:46:23.044ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2546 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 95650b45-a7ad-4cfc-8d70-79234ce5fd93 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16512024-12-14T04:46:24.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1a1272b9-24ca-4e30-bdf0-63e55bd51f77 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16522024-12-14T04:46:24.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1a1272b9-24ca-4e30-bdf0-63e55bd51f77 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16532024-12-14T04:46:24.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2526 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1a1272b9-24ca-4e30-bdf0-63e55bd51f77 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16542024-12-14T04:46:25.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b2ce9493-a8dc-4404-b44c-e092d99b0c74 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16552024-12-14T04:46:25.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b2ce9493-a8dc-4404-b44c-e092d99b0c74 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16562024-12-14T04:46:25.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1948 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b2ce9493-a8dc-4404-b44c-e092d99b0c74 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16572024-12-14T04:46:26.106ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c2f027ac-447d-47ec-88c2-770bbce3483d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16582024-12-14T04:46:26.107ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c2f027ac-447d-47ec-88c2-770bbce3483d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16592024-12-14T04:46:26.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2035 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c2f027ac-447d-47ec-88c2-770bbce3483d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16602024-12-14T04:46:26.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
16612024-12-14T04:46:26.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16622024-12-14T04:46:26.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16632024-12-14T04:46:26.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
16642024-12-14T04:46:26.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16652024-12-14T04:46:26.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16662024-12-14T04:46:26.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16672024-12-14T04:46:26.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16682024-12-14T04:46:26.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16692024-12-14T04:46:26.886ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.503318ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16702024-12-14T04:46:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16712024-12-14T04:46:26.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16722024-12-14T04:46:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16732024-12-14T04:46:26.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16742024-12-14T04:46:26.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16752024-12-14T04:46:26.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16762024-12-14T04:46:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
16772024-12-14T04:46:26.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16782024-12-14T04:46:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16792024-12-14T04:46:26.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
16802024-12-14T04:46:26.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.58636ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
16812024-12-14T04:46:27.128ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8dcc10c1-645d-4b66-baa1-e12986bb6b37 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16822024-12-14T04:46:27.128ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8dcc10c1-645d-4b66-baa1-e12986bb6b37 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16832024-12-14T04:46:27.131ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2735 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8dcc10c1-645d-4b66-baa1-e12986bb6b37 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16842024-12-14T04:46:28.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7270b986-eed7-4cf2-8111-077ae329e99c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16852024-12-14T04:46:28.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7270b986-eed7-4cf2-8111-077ae329e99c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16862024-12-14T04:46:28.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1857 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7270b986-eed7-4cf2-8111-077ae329e99c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16872024-12-14T04:46:29.173ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = feb4af16-4535-4bb3-bf56-4e47545affd7 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16882024-12-14T04:46:29.173ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = feb4af16-4535-4bb3-bf56-4e47545affd7 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16892024-12-14T04:46:29.175ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2242 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = feb4af16-4535-4bb3-bf56-4e47545affd7 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16902024-12-14T04:46:30.194ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5d2cd1fe-af64-4ca4-9e87-0506012ab050 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16912024-12-14T04:46:30.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5d2cd1fe-af64-4ca4-9e87-0506012ab050 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16922024-12-14T04:46:30.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2118 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5d2cd1fe-af64-4ca4-9e87-0506012ab050 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16932024-12-14T04:46:31.217ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7ce84b74-e35f-4d8f-ae7e-29e9e99c1b9f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16942024-12-14T04:46:31.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7ce84b74-e35f-4d8f-ae7e-29e9e99c1b9f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16952024-12-14T04:46:31.219ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2023 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7ce84b74-e35f-4d8f-ae7e-29e9e99c1b9f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16962024-12-14T04:46:32.239ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 57da4bd8-aeac-47c9-9968-bbb0e01d89e9 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16972024-12-14T04:46:32.239ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 57da4bd8-aeac-47c9-9968-bbb0e01d89e9 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16982024-12-14T04:46:32.241ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 57da4bd8-aeac-47c9-9968-bbb0e01d89e9 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
16992024-12-14T04:46:32.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1308 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = a7dccc7c-ed15-4e4d-9984-5d005c64fc88 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
17002024-12-14T04:46:33.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3696c82b-2058-4866-aa05-8ceebcaf9e8f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17012024-12-14T04:46:33.261ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3696c82b-2058-4866-aa05-8ceebcaf9e8f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17022024-12-14T04:46:33.263ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2510 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3696c82b-2058-4866-aa05-8ceebcaf9e8f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17032024-12-14T04:46:34.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 54451015-ff3b-4918-b079-7eae4bc21964 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17042024-12-14T04:46:34.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 54451015-ff3b-4918-b079-7eae4bc21964 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17052024-12-14T04:46:34.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2385 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 54451015-ff3b-4918-b079-7eae4bc21964 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17062024-12-14T04:46:35.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 739b10be-9a51-45ac-bedb-eb6a3eab30f1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17072024-12-14T04:46:35.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 739b10be-9a51-45ac-bedb-eb6a3eab30f1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17082024-12-14T04:46:35.307ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2377 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 739b10be-9a51-45ac-bedb-eb6a3eab30f1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17092024-12-14T04:46:36.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4e250a9e-e3b5-478d-bb77-851c459856b0 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17102024-12-14T04:46:36.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4e250a9e-e3b5-478d-bb77-851c459856b0 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17112024-12-14T04:46:36.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2054 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4e250a9e-e3b5-478d-bb77-851c459856b0 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
17122024-12-14T04:46:36.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = e545e613-57a3-4ebe-a36e-00bae93f812d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
17132024-12-14T04:46:36.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = e545e613-57a3-4ebe-a36e-00bae93f812d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
17142024-12-14T04:46:36.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2922 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = e545e613-57a3-4ebe-a36e-00bae93f812d response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
17152024-12-14T04:46:36.881ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
17162024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
17172024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17182024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17192024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
17202024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17212024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
17222024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
17232024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
17242024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
17252024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17262024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
17272024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
17282024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
17292024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
17302024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17312024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
17322024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
17332024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
17342024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17352024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
17362024-12-14T04:46:36.882ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
17372024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
17382024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17392024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17402024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
17412024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
17422024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17432024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17442024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
17452024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17462024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
17472024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17482024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
17492024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17502024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17512024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17522024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17532024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
17542024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17552024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17562024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17572024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17582024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17592024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17602024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17612024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
17622024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17632024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17642024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17652024-12-14T04:46:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
17662024-12-14T04:46:36.884ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17672024-12-14T04:46:36.885ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.058979ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
17682024-12-14T04:46:36.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17692024-12-14T04:46:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17702024-12-14T04:46:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17712024-12-14T04:46:36.886ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17722024-12-14T04:46:36.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1370, tv_nsec: 984112204 } })
17732024-12-14T04:46:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17742024-12-14T04:46:36.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1370, tv_nsec: 984233428 } })
17752024-12-14T04:46:36.886ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
17762024-12-14T04:46:36.887ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
17772024-12-14T04:46:36.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17782024-12-14T04:46:36.889ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17792024-12-14T04:46:36.889ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
17802024-12-14T04:46:36.889ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17812024-12-14T04:46:36.889ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17822024-12-14T04:46:36.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17832024-12-14T04:46:36.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17842024-12-14T04:46:36.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17852024-12-14T04:46:36.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17862024-12-14T04:46:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17872024-12-14T04:46:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17882024-12-14T04:46:36.891ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17892024-12-14T04:46:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17902024-12-14T04:46:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17912024-12-14T04:46:36.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17922024-12-14T04:46:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17932024-12-14T04:46:36.893ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17942024-12-14T04:46:36.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17952024-12-14T04:46:36.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17962024-12-14T04:46:36.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17972024-12-14T04:46:36.895ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17982024-12-14T04:46:36.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17992024-12-14T04:46:36.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002024-12-14T04:46:36.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18012024-12-14T04:46:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18022024-12-14T04:46:36.897ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18032024-12-14T04:46:36.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18042024-12-14T04:46:36.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18052024-12-14T04:46:36.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18062024-12-14T04:46:36.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18072024-12-14T04:46:36.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18082024-12-14T04:46:36.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18092024-12-14T04:46:36.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18102024-12-14T04:46:36.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
18112024-12-14T04:46:36.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18122024-12-14T04:46:36.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18132024-12-14T04:46:36.917ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1371, tv_nsec: 14443465 } })
18142024-12-14T04:46:36.917ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
18152024-12-14T04:46:36.919ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
18162024-12-14T04:46:36.919ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 37.264891ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
18172024-12-14T04:46:36.927ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 45.630479ms iteration = 28 status = {"physical_disks_added":0}
18182024-12-14T04:46:36.928ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
18192024-12-14T04:46:36.928ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.06316ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
18202024-12-14T04:46:36.928ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 46.607534ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
18212024-12-14T04:46:36.929ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18222024-12-14T04:46:36.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18232024-12-14T04:46:36.932ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242024-12-14T04:46:36.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
18252024-12-14T04:46:36.935ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18262024-12-14T04:46:36.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18272024-12-14T04:46:36.935ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
18282024-12-14T04:46:36.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
18292024-12-14T04:46:36.940ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 58.407152ms iteration = 24 status = {"errors":[],"records_set_to_done":[]}
18302024-12-14T04:46:36.948ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18312024-12-14T04:46:36.949ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18322024-12-14T04:46:36.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18332024-12-14T04:46:36.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18342024-12-14T04:46:36.952ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18352024-12-14T04:46:36.952ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
18362024-12-14T04:46:36.952ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18372024-12-14T04:46:36.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18382024-12-14T04:46:36.963ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
18392024-12-14T04:46:36.963ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18402024-12-14T04:46:36.963ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "16a9127b-2d55-43f2-8677-6c02dc6cd4ca", "content-length": "26", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
18412024-12-14T04:46:36.963ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18422024-12-14T04:46:36.964ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18432024-12-14T04:46:36.969ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
18442024-12-14T04:46:36.969ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18452024-12-14T04:46:36.969ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "945b126c-478b-45aa-b05d-ac66674bb48d", "content-length": "26", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
18462024-12-14T04:46:36.969ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18472024-12-14T04:46:36.971ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18482024-12-14T04:46:36.990ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18492024-12-14T04:46:36.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18502024-12-14T04:46:37.014ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
18512024-12-14T04:46:37.014ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18522024-12-14T04:46:37.015ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf7fee68-f589-44c9-8a2a-4090e40f2407", "content-length": "26", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
18532024-12-14T04:46:37.015ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18542024-12-14T04:46:37.016ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
18552024-12-14T04:46:37.139ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
18562024-12-14T04:46:37.139ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
18572024-12-14T04:46:37.139ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "095dce10-4f2d-4bfc-9fb8-c5dfc1e2464f", "content-length": "372", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
18582024-12-14T04:46:37.140ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18592024-12-14T04:46:37.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18602024-12-14T04:46:37.141ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 259.246795ms iteration = 24 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
18612024-12-14T04:46:37.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18622024-12-14T04:46:37.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18632024-12-14T04:46:37.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18642024-12-14T04:46:37.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18652024-12-14T04:46:37.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18662024-12-14T04:46:37.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18672024-12-14T04:46:37.173ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 291.29353ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
18682024-12-14T04:46:37.173ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18692024-12-14T04:46:37.173ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
18702024-12-14T04:46:37.173ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 291.788083ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
18712024-12-14T04:46:37.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 292.398779ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18722024-12-14T04:46:37.180ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 297.933913ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
18732024-12-14T04:46:37.181ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18742024-12-14T04:46:37.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18752024-12-14T04:46:37.183ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18762024-12-14T04:46:37.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
18772024-12-14T04:46:37.184ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18782024-12-14T04:46:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18792024-12-14T04:46:37.186ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18802024-12-14T04:46:37.187ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
18812024-12-14T04:46:37.187ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18822024-12-14T04:46:37.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18832024-12-14T04:46:37.191ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
18842024-12-14T04:46:37.191ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
18852024-12-14T04:46:37.191ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "224719e6-db17-417a-af74-54c481238fad", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
18862024-12-14T04:46:37.191ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18872024-12-14T04:46:37.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18882024-12-14T04:46:37.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18892024-12-14T04:46:37.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18902024-12-14T04:46:37.197ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18912024-12-14T04:46:37.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18922024-12-14T04:46:37.198ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18932024-12-14T04:46:37.199ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
18942024-12-14T04:46:37.199ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
18952024-12-14T04:46:37.200ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b6276362-d9a0-4176-9efa-61866d5965b3", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
18962024-12-14T04:46:37.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
18972024-12-14T04:46:37.200ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18982024-12-14T04:46:37.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18992024-12-14T04:46:37.202ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 320.09894ms iteration = 25 status = {}
19002024-12-14T04:46:37.204ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 322.740289ms iteration = 24 status = 0
19012024-12-14T04:46:37.229ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 347.443314ms iteration = 24 status = {}
19022024-12-14T04:46:37.232ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19032024-12-14T04:46:37.233ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "95039ffd-facf-4cd3-a9d8-39ca9f3f2989", "content-length": "1", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
19042024-12-14T04:46:37.234ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19052024-12-14T04:46:37.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19062024-12-14T04:46:37.236ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 354.597229ms iteration = 24 status = null
19072024-12-14T04:46:37.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19082024-12-14T04:46:37.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19092024-12-14T04:46:37.267ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
19102024-12-14T04:46:37.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19112024-12-14T04:46:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19122024-12-14T04:46:37.276ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19132024-12-14T04:46:37.277ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b49c067-2ab5-4da3-ad66-bc58a25dab6f", "content-length": "134", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
19142024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
19152024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
19162024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
19172024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
19182024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19192024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19202024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19212024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19222024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19232024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19242024-12-14T04:46:37.277ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
19252024-12-14T04:46:37.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19262024-12-14T04:46:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19272024-12-14T04:46:37.280ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19282024-12-14T04:46:37.280ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b691f21-5b88-43d0-b4a1-43238f11f8c1", "content-length": "2", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
19292024-12-14T04:46:37.280ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19302024-12-14T04:46:37.280ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc85bd0-4c4c-4e4d-b3ea-304e312c9a76", "content-length": "52", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
19312024-12-14T04:46:37.280ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
19322024-12-14T04:46:37.280ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19332024-12-14T04:46:37.283ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
19342024-12-14T04:46:37.373ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5dda52f2-a5fa-4265-8814-51e158a468ea", "date": "Sat, 14 Dec 2024 04:46:37 GMT"} })
19352024-12-14T04:46:37.373ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
19362024-12-14T04:46:37.373ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19372024-12-14T04:46:37.373ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a42788f-28df-4278-b8fa-5b616dee9ea5", "content-length": "691", "date": "Sat, 14 Dec 2024 04:46:37 GMT"} })
19382024-12-14T04:46:37.373ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
19392024-12-14T04:46:37.373ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
19402024-12-14T04:46:37.373ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
19412024-12-14T04:46:37.373ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c6051b4-8f9c-4edc-b311-8318776df5e4", "content-length": "682", "date": "Sat, 14 Dec 2024 04:46:36 GMT"} })
19422024-12-14T04:46:37.373ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19432024-12-14T04:46:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19442024-12-14T04:46:37.374ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 492.715018ms iteration = 24 status = {}
19452024-12-14T04:46:37.377ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19462024-12-14T04:46:37.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19472024-12-14T04:46:37.381ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19482024-12-14T04:46:37.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19492024-12-14T04:46:37.383ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19502024-12-14T04:46:37.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19512024-12-14T04:46:37.391ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 509.505436ms iteration = 24 status = {}
19522024-12-14T04:46:37.424ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3a69fbcf-5901-448c-9cb2-860f5c277a72 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19532024-12-14T04:46:37.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3a69fbcf-5901-448c-9cb2-860f5c277a72 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19542024-12-14T04:46:37.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2178 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3a69fbcf-5901-448c-9cb2-860f5c277a72 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19552024-12-14T04:46:38.443ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5b0fc9b5-7295-40cb-bfd8-8a653b1be729 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19562024-12-14T04:46:38.443ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5b0fc9b5-7295-40cb-bfd8-8a653b1be729 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19572024-12-14T04:46:38.445ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2378 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5b0fc9b5-7295-40cb-bfd8-8a653b1be729 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19582024-12-14T04:46:39.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c35a4b0-1abc-4a9c-b67a-aa42d519b16c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19592024-12-14T04:46:39.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c35a4b0-1abc-4a9c-b67a-aa42d519b16c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19602024-12-14T04:46:39.467ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2398 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3c35a4b0-1abc-4a9c-b67a-aa42d519b16c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19612024-12-14T04:46:40.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 666dc30f-cbe4-4a27-a33b-6f8ae9d72ccf roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19622024-12-14T04:46:40.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 666dc30f-cbe4-4a27-a33b-6f8ae9d72ccf resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19632024-12-14T04:46:40.490ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2568 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 666dc30f-cbe4-4a27-a33b-6f8ae9d72ccf response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19642024-12-14T04:46:41.509ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d339c71c-cba4-40b2-9e31-13f3fc62ecae roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19652024-12-14T04:46:41.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d339c71c-cba4-40b2-9e31-13f3fc62ecae resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19662024-12-14T04:46:41.511ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2472 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d339c71c-cba4-40b2-9e31-13f3fc62ecae response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19672024-12-14T04:46:42.530ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6c820c58-efc8-4dba-b7de-057a53fb6881 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19682024-12-14T04:46:42.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6c820c58-efc8-4dba-b7de-057a53fb6881 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19692024-12-14T04:46:42.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2363 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6c820c58-efc8-4dba-b7de-057a53fb6881 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19702024-12-14T04:46:42.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1357 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = f76131ec-703c-4177-9c79-4a29e0697097 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
19712024-12-14T04:46:43.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 027840a8-ee13-48ad-9246-9f2d475727c2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19722024-12-14T04:46:43.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 027840a8-ee13-48ad-9246-9f2d475727c2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19732024-12-14T04:46:43.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2246 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 027840a8-ee13-48ad-9246-9f2d475727c2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19742024-12-14T04:46:44.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a8219922-ba33-4d11-afb4-4b84247edad5 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19752024-12-14T04:46:44.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a8219922-ba33-4d11-afb4-4b84247edad5 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19762024-12-14T04:46:44.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2632 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a8219922-ba33-4d11-afb4-4b84247edad5 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19772024-12-14T04:46:45.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5ca1de4a-c946-48c2-bba9-d3c748599183 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19782024-12-14T04:46:45.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5ca1de4a-c946-48c2-bba9-d3c748599183 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19792024-12-14T04:46:45.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2801 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5ca1de4a-c946-48c2-bba9-d3c748599183 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19802024-12-14T04:46:46.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 67cd8975-6a95-42f1-98a9-d6adc30ecaa1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19812024-12-14T04:46:46.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 67cd8975-6a95-42f1-98a9-d6adc30ecaa1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19822024-12-14T04:46:46.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1958 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 67cd8975-6a95-42f1-98a9-d6adc30ecaa1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
19832024-12-14T04:46:46.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
19842024-12-14T04:46:46.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19852024-12-14T04:46:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19862024-12-14T04:46:46.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
19872024-12-14T04:46:46.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19882024-12-14T04:46:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19892024-12-14T04:46:46.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19902024-12-14T04:46:46.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19912024-12-14T04:46:46.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19922024-12-14T04:46:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19932024-12-14T04:46:46.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19942024-12-14T04:46:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19952024-12-14T04:46:46.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19962024-12-14T04:46:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19972024-12-14T04:46:46.915ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.872312ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19982024-12-14T04:46:46.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19992024-12-14T04:46:46.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
20002024-12-14T04:46:46.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20012024-12-14T04:46:46.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20022024-12-14T04:46:46.932ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
20032024-12-14T04:46:46.932ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.66932ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
20042024-12-14T04:46:47.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d207a18d-923c-4970-8563-eb7a17eb47ef roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20052024-12-14T04:46:47.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d207a18d-923c-4970-8563-eb7a17eb47ef resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20062024-12-14T04:46:47.646ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2148 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d207a18d-923c-4970-8563-eb7a17eb47ef response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20072024-12-14T04:46:47.769ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 8a8da585-e04d-4451-872f-2cabe32c027b roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
20082024-12-14T04:46:47.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 8a8da585-e04d-4451-872f-2cabe32c027b resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
20092024-12-14T04:46:47.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3087 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 8a8da585-e04d-4451-872f-2cabe32c027b response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
20102024-12-14T04:46:47.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = ed7262d2-3b95-4efa-97cb-aff283ad0881 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
20112024-12-14T04:46:47.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = ed7262d2-3b95-4efa-97cb-aff283ad0881 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
20122024-12-14T04:46:47.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2244 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = ed7262d2-3b95-4efa-97cb-aff283ad0881 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
20132024-12-14T04:46:48.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1eec7cbf-3915-4bf7-84f9-86ddc0f03f8e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20142024-12-14T04:46:48.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1eec7cbf-3915-4bf7-84f9-86ddc0f03f8e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20152024-12-14T04:46:48.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1838 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1eec7cbf-3915-4bf7-84f9-86ddc0f03f8e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20162024-12-14T04:46:49.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8a6d9cbe-526f-4ec9-b1d2-662190a6042f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20172024-12-14T04:46:49.684ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8a6d9cbe-526f-4ec9-b1d2-662190a6042f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20182024-12-14T04:46:49.686ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2066 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8a6d9cbe-526f-4ec9-b1d2-662190a6042f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20192024-12-14T04:46:50.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c021f1a0-768b-4f95-8108-7bb27d62c05d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20202024-12-14T04:46:50.704ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c021f1a0-768b-4f95-8108-7bb27d62c05d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20212024-12-14T04:46:50.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1919 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c021f1a0-768b-4f95-8108-7bb27d62c05d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20222024-12-14T04:46:51.725ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ccfe4da8-e3d9-4102-97cf-b3d944fb4f6d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20232024-12-14T04:46:51.725ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ccfe4da8-e3d9-4102-97cf-b3d944fb4f6d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20242024-12-14T04:46:51.727ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1968 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ccfe4da8-e3d9-4102-97cf-b3d944fb4f6d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20252024-12-14T04:46:52.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3a75108-63c3-4f5b-96bd-d539cfe44feb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20262024-12-14T04:46:52.748ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3a75108-63c3-4f5b-96bd-d539cfe44feb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20272024-12-14T04:46:52.750ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2817 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3a75108-63c3-4f5b-96bd-d539cfe44feb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20282024-12-14T04:46:52.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1504 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 6367a13b-a2d4-4084-8789-f41945989d56 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
20292024-12-14T04:46:53.771ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e20f7fc7-03ce-44bc-9f30-a85f9e352e93 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20302024-12-14T04:46:53.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e20f7fc7-03ce-44bc-9f30-a85f9e352e93 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20312024-12-14T04:46:53.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1418 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e20f7fc7-03ce-44bc-9f30-a85f9e352e93 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20322024-12-14T04:46:54.793ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9f0a3a38-e113-4702-b12a-5cfc7333463f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20332024-12-14T04:46:54.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9f0a3a38-e113-4702-b12a-5cfc7333463f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20342024-12-14T04:46:54.795ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1998 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9f0a3a38-e113-4702-b12a-5cfc7333463f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20352024-12-14T04:46:55.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = dc90865d-c0ea-4868-a01d-98b970cf2f3b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20362024-12-14T04:46:55.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = dc90865d-c0ea-4868-a01d-98b970cf2f3b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20372024-12-14T04:46:55.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2357 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = dc90865d-c0ea-4868-a01d-98b970cf2f3b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20382024-12-14T04:46:56.840ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 27abd6bd-36e9-435b-b1af-b6aef2570512 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20392024-12-14T04:46:56.841ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 27abd6bd-36e9-435b-b1af-b6aef2570512 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20402024-12-14T04:46:56.842ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2176 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 27abd6bd-36e9-435b-b1af-b6aef2570512 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20412024-12-14T04:46:56.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
20422024-12-14T04:46:56.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20432024-12-14T04:46:56.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20442024-12-14T04:46:56.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
20452024-12-14T04:46:56.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20462024-12-14T04:46:56.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20472024-12-14T04:46:56.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20482024-12-14T04:46:56.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20492024-12-14T04:46:56.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20502024-12-14T04:46:56.885ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.876051ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20512024-12-14T04:46:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20522024-12-14T04:46:56.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20532024-12-14T04:46:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20542024-12-14T04:46:56.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20552024-12-14T04:46:56.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20562024-12-14T04:46:56.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20572024-12-14T04:46:56.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
20582024-12-14T04:46:56.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20592024-12-14T04:46:56.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20602024-12-14T04:46:56.930ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
20612024-12-14T04:46:56.930ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.570496ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
20622024-12-14T04:46:57.861ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5b9b98bd-437e-4319-92cc-4411eaee2bab roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20632024-12-14T04:46:57.862ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5b9b98bd-437e-4319-92cc-4411eaee2bab resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20642024-12-14T04:46:57.863ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1471 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5b9b98bd-437e-4319-92cc-4411eaee2bab response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20652024-12-14T04:46:58.883ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1e2a068d-0ffe-448e-8337-46a368fff885 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20662024-12-14T04:46:58.883ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1e2a068d-0ffe-448e-8337-46a368fff885 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20672024-12-14T04:46:58.885ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2107 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1e2a068d-0ffe-448e-8337-46a368fff885 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20682024-12-14T04:46:59.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e574543-12cd-480a-ab64-cbf6700f2b12 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20692024-12-14T04:46:59.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e574543-12cd-480a-ab64-cbf6700f2b12 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20702024-12-14T04:46:59.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2075 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3e574543-12cd-480a-ab64-cbf6700f2b12 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20712024-12-14T04:47:00.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83a36864-c0b7-4a4d-a4ed-2be8a11161f3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20722024-12-14T04:47:00.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83a36864-c0b7-4a4d-a4ed-2be8a11161f3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20732024-12-14T04:47:00.928ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2303 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83a36864-c0b7-4a4d-a4ed-2be8a11161f3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20742024-12-14T04:47:01.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e33fb5c4-2972-49ca-b9f8-40c92ec078ef roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20752024-12-14T04:47:01.949ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e33fb5c4-2972-49ca-b9f8-40c92ec078ef resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20762024-12-14T04:47:01.950ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2041 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e33fb5c4-2972-49ca-b9f8-40c92ec078ef response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20772024-12-14T04:47:02.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1733 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 07a9cc77-e5c1-4592-bd6a-3e9f5db35d79 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
20782024-12-14T04:47:02.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 028c5148-8d9e-46f0-8867-4ab2e16ae879 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20792024-12-14T04:47:02.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 028c5148-8d9e-46f0-8867-4ab2e16ae879 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20802024-12-14T04:47:02.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1583 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 028c5148-8d9e-46f0-8867-4ab2e16ae879 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20812024-12-14T04:47:03.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e37ce584-1d14-4e1e-919f-7de71e2fd9eb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20822024-12-14T04:47:03.993ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e37ce584-1d14-4e1e-919f-7de71e2fd9eb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20832024-12-14T04:47:03.995ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2866 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e37ce584-1d14-4e1e-919f-7de71e2fd9eb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20842024-12-14T04:47:05.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9e0c1bc0-caab-4d7a-bf7c-6313bd84dd63 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20852024-12-14T04:47:05.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9e0c1bc0-caab-4d7a-bf7c-6313bd84dd63 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20862024-12-14T04:47:05.018ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3328 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9e0c1bc0-caab-4d7a-bf7c-6313bd84dd63 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20872024-12-14T04:47:06.047ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 69f23a1a-0adf-4998-9978-eb4a44638f4f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20882024-12-14T04:47:06.048ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 69f23a1a-0adf-4998-9978-eb4a44638f4f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20892024-12-14T04:47:06.049ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2374 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 69f23a1a-0adf-4998-9978-eb4a44638f4f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
20902024-12-14T04:47:06.729ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 6474d055-28d2-41d8-b2c2-71f97e9000d3 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
20912024-12-14T04:47:06.729ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 6474d055-28d2-41d8-b2c2-71f97e9000d3 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
20922024-12-14T04:47:06.733ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4594 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 6474d055-28d2-41d8-b2c2-71f97e9000d3 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
20932024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
20942024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
20952024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
20962024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
20972024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
20982024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20992024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
21002024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
21012024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
21022024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21032024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
21042024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
21052024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21062024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
21072024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
21082024-12-14T04:47:06.882ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 04:37:06.882208355 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
21092024-12-14T04:47:06.882ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
21102024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21112024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
21122024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
21132024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
21142024-12-14T04:47:06.882ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832
21152024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.76\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832"}
21162024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
21172024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21182024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
21192024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
21202024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
21212024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21222024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
21232024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
21242024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21252024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
21262024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21272024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
21282024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
21292024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
21302024-12-14T04:47:06.882ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
21312024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21322024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
21332024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
21342024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
21352024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
21362024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21372024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
21382024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21392024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
21402024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
21412024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21422024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21432024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
21442024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
21452024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21462024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
21472024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21482024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
21492024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21502024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
21512024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
21522024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21532024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21542024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
21552024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
21562024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
21572024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
21582024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
21592024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21602024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21612024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21622024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21632024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21642024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21652024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
21662024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
21672024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21682024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21692024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
21702024-12-14T04:47:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21712024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21722024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21732024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21742024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21752024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21762024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
21772024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
21782024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21792024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21802024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21812024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21822024-12-14T04:47:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2224041a-a9fc-46d2-b9ed-ef98dc01f696.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("229c9d57-e318-46f6-8df9-9f602a126f12.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1400, tv_nsec: 980464119 } })
21832024-12-14T04:47:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ba1572d-df24-4bd2-be03-9a99edc867d8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1400, tv_nsec: 980471070 } })
21842024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21852024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21862024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21872024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21882024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21892024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21902024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21912024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21922024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
21932024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.496609ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
21942024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
21952024-12-14T04:47:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.123579ms iteration = 14 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
21962024-12-14T04:47:06.885ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.089521ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
21972024-12-14T04:47:06.885ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21982024-12-14T04:47:06.885ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.576724ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
21992024-12-14T04:47:06.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1400, tv_nsec: 983975251 } })
22002024-12-14T04:47:06.886ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
22012024-12-14T04:47:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
22022024-12-14T04:47:06.887ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
22032024-12-14T04:47:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
22042024-12-14T04:47:06.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22052024-12-14T04:47:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22062024-12-14T04:47:06.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22072024-12-14T04:47:06.889ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1400, tv_nsec: 987293447 } })
22082024-12-14T04:47:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22092024-12-14T04:47:06.890ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
22102024-12-14T04:47:06.891ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 04:37:06.882208355 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
22112024-12-14T04:47:06.891ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.214691ms iteration = 13 status = {"expiration":"2024-12-14T04:37:06.882208355Z","pruned":[]}
22122024-12-14T04:47:06.932ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22132024-12-14T04:47:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22142024-12-14T04:47:06.933ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.146452ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
22152024-12-14T04:47:06.935ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22162024-12-14T04:47:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
22172024-12-14T04:47:06.938ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
22182024-12-14T04:47:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
22192024-12-14T04:47:06.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22202024-12-14T04:47:06.940ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22212024-12-14T04:47:06.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22222024-12-14T04:47:06.940ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22232024-12-14T04:47:06.941ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22242024-12-14T04:47:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22252024-12-14T04:47:06.943ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22262024-12-14T04:47:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22272024-12-14T04:47:06.943ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22282024-12-14T04:47:06.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
22292024-12-14T04:47:06.944ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
22302024-12-14T04:47:06.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
22312024-12-14T04:47:06.944ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 62.578591ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
22322024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22332024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
22342024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
22352024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
22362024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22372024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22382024-12-14T04:47:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22392024-12-14T04:47:06.947ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22402024-12-14T04:47:06.947ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22412024-12-14T04:47:06.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
22422024-12-14T04:47:06.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22432024-12-14T04:47:06.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22442024-12-14T04:47:06.948ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22452024-12-14T04:47:06.949ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22462024-12-14T04:47:06.950ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22472024-12-14T04:47:06.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22482024-12-14T04:47:06.950ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22492024-12-14T04:47:06.950ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
22502024-12-14T04:47:06.950ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22512024-12-14T04:47:06.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22522024-12-14T04:47:06.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22532024-12-14T04:47:06.951ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22542024-12-14T04:47:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
22552024-12-14T04:47:06.951ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
22562024-12-14T04:47:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22572024-12-14T04:47:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
22582024-12-14T04:47:06.952ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22592024-12-14T04:47:06.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22602024-12-14T04:47:06.952ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22612024-12-14T04:47:06.952ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
22622024-12-14T04:47:06.952ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
22632024-12-14T04:47:06.952ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 70.656511ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
22642024-12-14T04:47:06.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22652024-12-14T04:47:06.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22662024-12-14T04:47:06.954ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22672024-12-14T04:47:06.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22682024-12-14T04:47:06.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22692024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22702024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 72.774217ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
22712024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
22722024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
22732024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22742024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
22752024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22762024-12-14T04:47:06.955ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22772024-12-14T04:47:06.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22782024-12-14T04:47:06.956ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1401, tv_nsec: 54324414 } })
22792024-12-14T04:47:06.957ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
22802024-12-14T04:47:06.958ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22812024-12-14T04:47:06.960ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
22822024-12-14T04:47:06.960ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22832024-12-14T04:47:06.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22842024-12-14T04:47:06.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22852024-12-14T04:47:06.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22862024-12-14T04:47:06.966ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22872024-12-14T04:47:06.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22882024-12-14T04:47:06.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22892024-12-14T04:47:06.968ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22902024-12-14T04:47:06.968ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22912024-12-14T04:47:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22922024-12-14T04:47:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22932024-12-14T04:47:06.971ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 89.183884ms iteration = 6 status = {"nsuccess":5}
22942024-12-14T04:47:06.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
22952024-12-14T04:47:06.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22962024-12-14T04:47:06.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22972024-12-14T04:47:06.974ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22982024-12-14T04:47:06.975ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22992024-12-14T04:47:06.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23002024-12-14T04:47:06.978ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23012024-12-14T04:47:06.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23022024-12-14T04:47:06.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
23032024-12-14T04:47:06.979ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23042024-12-14T04:47:06.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23052024-12-14T04:47:06.984ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23062024-12-14T04:47:06.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
23072024-12-14T04:47:06.987ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23082024-12-14T04:47:06.987ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
23092024-12-14T04:47:06.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23102024-12-14T04:47:06.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
23112024-12-14T04:47:06.992ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
23122024-12-14T04:47:06.992ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 110.062308ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
23132024-12-14T04:47:06.993ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
23142024-12-14T04:47:06.993ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
23152024-12-14T04:47:06.993ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
23162024-12-14T04:47:06.993ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23172024-12-14T04:47:06.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23182024-12-14T04:47:06.995ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23192024-12-14T04:47:06.996ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
23202024-12-14T04:47:06.996ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
23212024-12-14T04:47:06.996ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
23222024-12-14T04:47:06.996ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23232024-12-14T04:47:06.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23242024-12-14T04:47:06.998ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23252024-12-14T04:47:06.999ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23262024-12-14T04:47:06.999ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23272024-12-14T04:47:06.999ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
23282024-12-14T04:47:06.999ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
23292024-12-14T04:47:06.999ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 116.959657ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
23302024-12-14T04:47:06.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23312024-12-14T04:47:07.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23322024-12-14T04:47:07.001ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23332024-12-14T04:47:07.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23342024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
23352024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 0 version = 2
23362024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 119.714139ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23372024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23382024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23392024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23402024-12-14T04:47:07.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23412024-12-14T04:47:07.004ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
23422024-12-14T04:47:07.004ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
23432024-12-14T04:47:07.004ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23442024-12-14T04:47:07.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23452024-12-14T04:47:07.005ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23462024-12-14T04:47:07.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23472024-12-14T04:47:07.008ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23482024-12-14T04:47:07.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23492024-12-14T04:47:07.013ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23502024-12-14T04:47:07.013ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23512024-12-14T04:47:07.013ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23522024-12-14T04:47:07.014ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23532024-12-14T04:47:07.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23542024-12-14T04:47:07.014ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23552024-12-14T04:47:07.014ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
23562024-12-14T04:47:07.014ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23572024-12-14T04:47:07.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23582024-12-14T04:47:07.014ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 132.661776ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23592024-12-14T04:47:07.016ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
23602024-12-14T04:47:07.016ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23612024-12-14T04:47:07.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23622024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23632024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23642024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
23652024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23662024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23672024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23682024-12-14T04:47:07.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23692024-12-14T04:47:07.020ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23702024-12-14T04:47:07.021ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 139.148686ms iteration = 25 status = 0
23712024-12-14T04:47:07.022ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 1 version = 2
23722024-12-14T04:47:07.022ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
23732024-12-14T04:47:07.022ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 04:35:19.721992 UTC dns_group = external generation = 2
23742024-12-14T04:47:07.022ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 139.787923ms iteration = 14 status = {"generation":2}
23752024-12-14T04:47:07.022ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23762024-12-14T04:47:07.022ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23772024-12-14T04:47:07.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23782024-12-14T04:47:07.027ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
23792024-12-14T04:47:07.027ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
23802024-12-14T04:47:07.027ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 04:35:18.025163 UTC dns_group = internal generation = 1
23812024-12-14T04:47:07.027ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 145.153712ms iteration = 14 status = {"generation":1}
23822024-12-14T04:47:07.034ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
23832024-12-14T04:47:07.034ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23842024-12-14T04:47:07.035ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ed7d14-488b-42a7-8cc6-ef3eb0a2c3cc", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
23852024-12-14T04:47:07.035ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23862024-12-14T04:47:07.037ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23872024-12-14T04:47:07.038ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
23882024-12-14T04:47:07.038ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 155.692147ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
23892024-12-14T04:47:07.040ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23902024-12-14T04:47:07.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23912024-12-14T04:47:07.057ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 175.426151ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
23922024-12-14T04:47:07.059ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23932024-12-14T04:47:07.059ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
23942024-12-14T04:47:07.059ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 176.954591ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
23952024-12-14T04:47:07.062ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
23962024-12-14T04:47:07.062ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 179.525928ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
23972024-12-14T04:47:07.069ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T04:35:03.711237Z, time_modified: 2024-12-14T04:35:03.711237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }) }
23982024-12-14T04:47:07.069ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 187.177238ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]}},"default_endpoint":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
23992024-12-14T04:47:07.087ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5114f9e1-1d2f-4ebc-bdbc-a2582cda40af roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
24002024-12-14T04:47:07.088ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5114f9e1-1d2f-4ebc-bdbc-a2582cda40af resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
24012024-12-14T04:47:07.089ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1887 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5114f9e1-1d2f-4ebc-bdbc-a2582cda40af response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
24022024-12-14T04:47:07.096ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 213.949095ms iteration = 29 status = {"physical_disks_added":0}
24032024-12-14T04:47:07.130ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
24042024-12-14T04:47:07.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24052024-12-14T04:47:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24062024-12-14T04:47:07.147ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24072024-12-14T04:47:07.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24082024-12-14T04:47:07.149ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24092024-12-14T04:47:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24102024-12-14T04:47:07.166ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
24112024-12-14T04:47:07.166ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24122024-12-14T04:47:07.167ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "85b18070-d0f9-4bea-9990-129a541e042a", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24132024-12-14T04:47:07.167ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24142024-12-14T04:47:07.169ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24152024-12-14T04:47:07.172ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
24162024-12-14T04:47:07.172ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24172024-12-14T04:47:07.172ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fd9647f-b8f2-4b6e-af21-eb3fb4aaae12", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24182024-12-14T04:47:07.172ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24192024-12-14T04:47:07.175ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
24202024-12-14T04:47:07.177ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24212024-12-14T04:47:07.179ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24222024-12-14T04:47:07.179ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24232024-12-14T04:47:07.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24242024-12-14T04:47:07.204ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
24252024-12-14T04:47:07.204ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "52a024f0-a0d0-4875-b6e7-4d1a62afb1a0", "content-length": "1", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24262024-12-14T04:47:07.206ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
24272024-12-14T04:47:07.206ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9f8c7e7b-5d47-4320-959f-35ab593f5471", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24282024-12-14T04:47:07.206ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24292024-12-14T04:47:07.206ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24302024-12-14T04:47:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24312024-12-14T04:47:07.208ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "72f88380-61b2-456d-9c57-d73aad42d0d1", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24322024-12-14T04:47:07.208ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 326.154999ms iteration = 25 status = null
24332024-12-14T04:47:07.208ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24342024-12-14T04:47:07.210ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24352024-12-14T04:47:07.210ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 328.14782ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
24362024-12-14T04:47:07.261ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
24372024-12-14T04:47:07.261ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24382024-12-14T04:47:07.262ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "27bbc648-c7a9-43f5-8817-ac005701b19f", "content-length": "372", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24392024-12-14T04:47:07.262ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24402024-12-14T04:47:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24412024-12-14T04:47:07.302ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
24422024-12-14T04:47:07.302ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
24432024-12-14T04:47:07.303ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "63e23a20-ff00-44fe-adf4-58ebe448d531", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24442024-12-14T04:47:07.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24452024-12-14T04:47:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24462024-12-14T04:47:07.305ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 423.074315ms iteration = 25 status = {}
24472024-12-14T04:47:07.305ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24482024-12-14T04:47:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24492024-12-14T04:47:07.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24502024-12-14T04:47:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24512024-12-14T04:47:07.311ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
24522024-12-14T04:47:07.311ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24532024-12-14T04:47:07.312ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d982b0da-f084-4e5e-9eca-6cd7354d0285", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24542024-12-14T04:47:07.313ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 431.421402ms iteration = 26 status = {}
24552024-12-14T04:47:07.344ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
24562024-12-14T04:47:07.344ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e85df84c-71b2-474a-ac6e-d00f7b538403", "content-length": "691", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24572024-12-14T04:47:07.344ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
24582024-12-14T04:47:07.344ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
24592024-12-14T04:47:07.344ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
24602024-12-14T04:47:07.345ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 463.22936ms iteration = 25 status = {}
24612024-12-14T04:47:07.355ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
24622024-12-14T04:47:07.464ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
24632024-12-14T04:47:07.464ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1c47b6f6-fdbb-43b9-8b39-762d5704381d", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24642024-12-14T04:47:07.464ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24652024-12-14T04:47:07.467ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1213dec3-565e-4b97-877d-433e104e4b1a", "date": "Sat, 14 Dec 2024 04:47:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24662024-12-14T04:47:07.467ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24672024-12-14T04:47:07.468ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "f7541cd6-e864-4aca-add3-2d147ea3d9bb", "date": "Sat, 14 Dec 2024 04:47:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24682024-12-14T04:47:07.468ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24692024-12-14T04:47:07.470ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
24702024-12-14T04:47:07.470ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 588.567506ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
24712024-12-14T04:47:07.501ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24722024-12-14T04:47:07.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24732024-12-14T04:47:07.503ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
24742024-12-14T04:47:07.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24752024-12-14T04:47:07.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24762024-12-14T04:47:07.511ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
24772024-12-14T04:47:07.512ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad2050ae-36d4-41c5-aff7-72b5e48fa8e7", "content-length": "134", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24782024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
24792024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
24802024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
24812024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
24822024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24832024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24842024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24852024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24862024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24872024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24882024-12-14T04:47:07.512ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
24892024-12-14T04:47:07.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24902024-12-14T04:47:07.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24912024-12-14T04:47:07.515ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
24922024-12-14T04:47:07.515ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b2bf97e-3d93-4b17-adee-237372a070f6", "content-length": "2", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24932024-12-14T04:47:07.515ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
24942024-12-14T04:47:07.515ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f56d9102-cacb-4253-91f2-0c6e4a18ba64", "content-length": "52", "date": "Sat, 14 Dec 2024 04:47:06 GMT"} })
24952024-12-14T04:47:07.515ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
24962024-12-14T04:47:07.515ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24972024-12-14T04:47:07.615ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bc1f2b13-b873-429c-88fe-6f377bd3d651", "date": "Sat, 14 Dec 2024 04:47:07 GMT"} })
24982024-12-14T04:47:07.615ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
24992024-12-14T04:47:07.615ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
25002024-12-14T04:47:07.615ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fff78b10-241c-45c4-a6eb-f864e26e1247", "content-length": "682", "date": "Sat, 14 Dec 2024 04:47:07 GMT"} })
25012024-12-14T04:47:07.615ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25022024-12-14T04:47:07.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25032024-12-14T04:47:07.619ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25042024-12-14T04:47:07.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25052024-12-14T04:47:07.622ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25062024-12-14T04:47:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25072024-12-14T04:47:07.625ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25082024-12-14T04:47:07.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25092024-12-14T04:47:07.632ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 750.233528ms iteration = 25 status = {}
25102024-12-14T04:47:08.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f7e5cd07-2285-4647-83b6-9efbda15ec7e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25112024-12-14T04:47:08.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f7e5cd07-2285-4647-83b6-9efbda15ec7e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25122024-12-14T04:47:08.116ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2858 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f7e5cd07-2285-4647-83b6-9efbda15ec7e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25132024-12-14T04:47:09.136ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af8ba9d9-9a71-4a3e-ad11-ba3ff872aa9c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25142024-12-14T04:47:09.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af8ba9d9-9a71-4a3e-ad11-ba3ff872aa9c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25152024-12-14T04:47:09.138ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1985 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = af8ba9d9-9a71-4a3e-ad11-ba3ff872aa9c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25162024-12-14T04:47:10.158ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6e4cc762-3c12-4268-98ca-a4c18e8cbb1c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25172024-12-14T04:47:10.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6e4cc762-3c12-4268-98ca-a4c18e8cbb1c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25182024-12-14T04:47:10.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2326 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6e4cc762-3c12-4268-98ca-a4c18e8cbb1c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25192024-12-14T04:47:11.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5e7f3f41-a907-4080-85d9-831f34477796 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25202024-12-14T04:47:11.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5e7f3f41-a907-4080-85d9-831f34477796 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25212024-12-14T04:47:11.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2699 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5e7f3f41-a907-4080-85d9-831f34477796 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25222024-12-14T04:47:12.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 40870a94-7362-4760-9a9b-a3450ff46b0d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25232024-12-14T04:47:12.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 40870a94-7362-4760-9a9b-a3450ff46b0d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25242024-12-14T04:47:12.206ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2300 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 40870a94-7362-4760-9a9b-a3450ff46b0d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25252024-12-14T04:47:12.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1272 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 69acbfac-9229-4315-9db8-aa96dfc71d9a response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
25262024-12-14T04:47:13.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e45ce65c-059a-4ee8-b156-6a2a9edd2634 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25272024-12-14T04:47:13.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e45ce65c-059a-4ee8-b156-6a2a9edd2634 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25282024-12-14T04:47:13.227ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2074 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e45ce65c-059a-4ee8-b156-6a2a9edd2634 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25292024-12-14T04:47:14.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5856ec3d-8d82-4bc9-80d1-c7849ebb6880 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25302024-12-14T04:47:14.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5856ec3d-8d82-4bc9-80d1-c7849ebb6880 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25312024-12-14T04:47:14.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1603 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5856ec3d-8d82-4bc9-80d1-c7849ebb6880 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25322024-12-14T04:47:14.742ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 0bf0442f-ec4a-4f13-9ef4-f47c12950efe roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
25332024-12-14T04:47:14.743ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 0bf0442f-ec4a-4f13-9ef4-f47c12950efe resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
25342024-12-14T04:47:14.745ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2601 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 0bf0442f-ec4a-4f13-9ef4-f47c12950efe response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
25352024-12-14T04:47:14.746ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = e19443cd-3043-4070-b64e-dc2ce97d5855 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
25362024-12-14T04:47:14.746ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = e19443cd-3043-4070-b64e-dc2ce97d5855 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
25372024-12-14T04:47:14.749ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3146 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = e19443cd-3043-4070-b64e-dc2ce97d5855 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
25382024-12-14T04:47:15.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b5f96740-814f-4eba-84cf-4bbf0b95dcc8 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25392024-12-14T04:47:15.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b5f96740-814f-4eba-84cf-4bbf0b95dcc8 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25402024-12-14T04:47:15.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2647 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b5f96740-814f-4eba-84cf-4bbf0b95dcc8 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25412024-12-14T04:47:16.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a1b9e795-5749-4417-8125-16282a93a132 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25422024-12-14T04:47:16.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a1b9e795-5749-4417-8125-16282a93a132 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25432024-12-14T04:47:16.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2556 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a1b9e795-5749-4417-8125-16282a93a132 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25442024-12-14T04:47:16.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
25452024-12-14T04:47:16.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25462024-12-14T04:47:16.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25472024-12-14T04:47:16.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
25482024-12-14T04:47:16.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25492024-12-14T04:47:16.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25502024-12-14T04:47:16.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25512024-12-14T04:47:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25522024-12-14T04:47:16.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25532024-12-14T04:47:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25542024-12-14T04:47:16.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25552024-12-14T04:47:16.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25562024-12-14T04:47:16.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25572024-12-14T04:47:16.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
25582024-12-14T04:47:16.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25592024-12-14T04:47:16.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25602024-12-14T04:47:16.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
25612024-12-14T04:47:16.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.839371ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
25622024-12-14T04:47:16.913ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25632024-12-14T04:47:16.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25642024-12-14T04:47:16.948ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.909167ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25652024-12-14T04:47:17.316ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e178b2c2-062a-45d5-b799-77a3e422708d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25662024-12-14T04:47:17.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e178b2c2-062a-45d5-b799-77a3e422708d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25672024-12-14T04:47:17.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3147 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e178b2c2-062a-45d5-b799-77a3e422708d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25682024-12-14T04:47:18.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6c594ae7-8bc0-446a-892f-bc247460ec67 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25692024-12-14T04:47:18.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6c594ae7-8bc0-446a-892f-bc247460ec67 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25702024-12-14T04:47:18.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2245 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6c594ae7-8bc0-446a-892f-bc247460ec67 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25712024-12-14T04:47:19.373ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 27f7ac30-b8c0-4f48-8c95-53d9a96ec4bb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25722024-12-14T04:47:19.374ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 27f7ac30-b8c0-4f48-8c95-53d9a96ec4bb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25732024-12-14T04:47:19.376ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2261 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 27f7ac30-b8c0-4f48-8c95-53d9a96ec4bb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25742024-12-14T04:47:20.395ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f47dd0ee-f2d1-4325-9839-45d8af965dc8 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25752024-12-14T04:47:20.396ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f47dd0ee-f2d1-4325-9839-45d8af965dc8 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25762024-12-14T04:47:20.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2396 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f47dd0ee-f2d1-4325-9839-45d8af965dc8 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25772024-12-14T04:47:21.416ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 89e1fa9f-1293-4946-a191-a714858719b8 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25782024-12-14T04:47:21.416ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 89e1fa9f-1293-4946-a191-a714858719b8 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25792024-12-14T04:47:21.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2510 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 89e1fa9f-1293-4946-a191-a714858719b8 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25802024-12-14T04:47:22.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6215a013-6c80-4dba-8619-6542fa56edea roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25812024-12-14T04:47:22.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6215a013-6c80-4dba-8619-6542fa56edea resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25822024-12-14T04:47:22.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2364 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6215a013-6c80-4dba-8619-6542fa56edea response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25832024-12-14T04:47:22.595ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d988feca-f58d-4b50-9264-78eee1ede440 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-14T04:47:22.589813002Z }
25842024-12-14T04:47:22.595ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d988feca-f58d-4b50-9264-78eee1ede440 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
25852024-12-14T04:47:22.595ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d988feca-f58d-4b50-9264-78eee1ede440 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
25862024-12-14T04:47:22.624ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1:0]:37743
25872024-12-14T04:47:22.624ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:37743 req_id = a6b0f059-2130-4c6c-ba6d-635b0a38e866 roles = RoleSet { roles: {} } uri = /metrics/producers
25882024-12-14T04:47:22.624ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:37743 req_id = a6b0f059-2130-4c6c-ba6d-635b0a38e866 resource = Database result = Ok(()) uri = /metrics/producers
25892024-12-14T04:47:22.636ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46279 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d988feca-f58d-4b50-9264-78eee1ede440 response_code = 204 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
25902024-12-14T04:47:22.637ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d2d3ecc6-f900-4768-b0cc-9deee1df13fb uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-14T04:47:22.636879218Z }
25912024-12-14T04:47:22.637ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d2d3ecc6-f900-4768-b0cc-9deee1df13fb roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
25922024-12-14T04:47:22.637ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d2d3ecc6-f900-4768-b0cc-9deee1df13fb resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
25932024-12-14T04:47:22.664ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3
25942024-12-14T04:47:22.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40317 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:37743 req_id = a6b0f059-2130-4c6c-ba6d-635b0a38e866 response_code = 201 uri = /metrics/producers
25952024-12-14T04:47:22.682ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45532 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d2d3ecc6-f900-4768-b0cc-9deee1df13fb response_code = 204 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
25962024-12-14T04:47:22.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1279 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 535aaa9e-386d-4ad5-a9a2-1f958d40e1ad response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
25972024-12-14T04:47:23.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c3ac90c7-19f8-4711-a88f-951c34dcaa0d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25982024-12-14T04:47:23.463ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c3ac90c7-19f8-4711-a88f-951c34dcaa0d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
25992024-12-14T04:47:23.465ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2547 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c3ac90c7-19f8-4711-a88f-951c34dcaa0d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26002024-12-14T04:47:24.486ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 993c91b5-636c-445c-9a24-8d060380c9dc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26012024-12-14T04:47:24.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 993c91b5-636c-445c-9a24-8d060380c9dc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26022024-12-14T04:47:24.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2722 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 993c91b5-636c-445c-9a24-8d060380c9dc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26032024-12-14T04:47:25.509ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4fa9d638-dbc9-4ac0-9b56-a22b492f3ba0 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26042024-12-14T04:47:25.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4fa9d638-dbc9-4ac0-9b56-a22b492f3ba0 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26052024-12-14T04:47:25.511ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2411 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4fa9d638-dbc9-4ac0-9b56-a22b492f3ba0 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26062024-12-14T04:47:26.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 91819982-49a2-46a6-b946-3811dbc5ce0b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26072024-12-14T04:47:26.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 91819982-49a2-46a6-b946-3811dbc5ce0b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26082024-12-14T04:47:26.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1811 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 91819982-49a2-46a6-b946-3811dbc5ce0b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26092024-12-14T04:47:26.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
26102024-12-14T04:47:26.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26112024-12-14T04:47:26.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26122024-12-14T04:47:26.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
26132024-12-14T04:47:26.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26142024-12-14T04:47:26.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26152024-12-14T04:47:26.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26162024-12-14T04:47:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26172024-12-14T04:47:26.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26182024-12-14T04:47:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26192024-12-14T04:47:26.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26202024-12-14T04:47:26.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26212024-12-14T04:47:26.912ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26222024-12-14T04:47:26.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26232024-12-14T04:47:26.914ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.735433ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26242024-12-14T04:47:26.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26252024-12-14T04:47:26.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
26262024-12-14T04:47:26.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26272024-12-14T04:47:26.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26282024-12-14T04:47:26.933ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
26292024-12-14T04:47:26.933ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.176087ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
26302024-12-14T04:47:27.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 366d9062-c0fd-4999-b7a2-e62567c7c90b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26312024-12-14T04:47:27.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 366d9062-c0fd-4999-b7a2-e62567c7c90b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26322024-12-14T04:47:27.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2429 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 366d9062-c0fd-4999-b7a2-e62567c7c90b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26332024-12-14T04:47:28.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 674448e8-9b02-4601-b6b8-9d3718b41b81 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26342024-12-14T04:47:28.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 674448e8-9b02-4601-b6b8-9d3718b41b81 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26352024-12-14T04:47:28.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2384 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 674448e8-9b02-4601-b6b8-9d3718b41b81 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26362024-12-14T04:47:29.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1888bed5-523a-4a8a-b8e7-40e3cce40308 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26372024-12-14T04:47:29.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1888bed5-523a-4a8a-b8e7-40e3cce40308 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26382024-12-14T04:47:29.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2366 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1888bed5-523a-4a8a-b8e7-40e3cce40308 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26392024-12-14T04:47:30.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 65df63e1-f76c-4b23-a865-8301d542828c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26402024-12-14T04:47:30.623ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 65df63e1-f76c-4b23-a865-8301d542828c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26412024-12-14T04:47:30.624ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1827 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 65df63e1-f76c-4b23-a865-8301d542828c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26422024-12-14T04:47:31.644ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d05417e0-1f17-4653-a789-bdbf8de76f71 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26432024-12-14T04:47:31.645ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d05417e0-1f17-4653-a789-bdbf8de76f71 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26442024-12-14T04:47:31.646ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2276 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d05417e0-1f17-4653-a789-bdbf8de76f71 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26452024-12-14T04:47:32.192ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2d9cd54c-026c-411a-a658-be1ab3f7bcc1 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-14T04:47:32.191735109Z }
26462024-12-14T04:47:32.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2d9cd54c-026c-411a-a658-be1ab3f7bcc1 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26472024-12-14T04:47:32.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2d9cd54c-026c-411a-a658-be1ab3f7bcc1 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26482024-12-14T04:47:32.264ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 16 transaction = vmm_and_migration_update_runtime
26492024-12-14T04:47:32.290ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = vmm_and_migration_update_runtime
26502024-12-14T04:47:32.290ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98273 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2d9cd54c-026c-411a-a658-be1ab3f7bcc1 response_code = 204 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26512024-12-14T04:47:32.667ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7d8f7fee-2864-4b31-9517-bc7422c1d58f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26522024-12-14T04:47:32.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7d8f7fee-2864-4b31-9517-bc7422c1d58f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26532024-12-14T04:47:32.669ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2128 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7d8f7fee-2864-4b31-9517-bc7422c1d58f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26542024-12-14T04:47:32.769ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 4a17f6a9-2be3-41d3-a4c3-266a0247c6ff roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
26552024-12-14T04:47:32.769ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 4a17f6a9-2be3-41d3-a4c3-266a0247c6ff resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
26562024-12-14T04:47:32.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3842 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 4a17f6a9-2be3-41d3-a4c3-266a0247c6ff response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
26572024-12-14T04:47:32.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 131dcc8e-297d-4581-be4f-8ec457c34dee roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
26582024-12-14T04:47:32.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 131dcc8e-297d-4581-be4f-8ec457c34dee resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
26592024-12-14T04:47:32.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3390 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 131dcc8e-297d-4581-be4f-8ec457c34dee response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
26602024-12-14T04:47:32.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1367 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 1b8f3eb9-3528-4571-90c5-5067aa751947 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
26612024-12-14T04:47:33.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9f43b61c-eb05-4ae6-bcff-ea898e71115c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26622024-12-14T04:47:33.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9f43b61c-eb05-4ae6-bcff-ea898e71115c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26632024-12-14T04:47:33.694ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2192 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9f43b61c-eb05-4ae6-bcff-ea898e71115c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26642024-12-14T04:47:33.777ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 11654ac1-4933-4cba-8de5-4d552f90cab2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
26652024-12-14T04:47:33.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 11654ac1-4933-4cba-8de5-4d552f90cab2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
26662024-12-14T04:47:33.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2752 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 11654ac1-4933-4cba-8de5-4d552f90cab2 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
26672024-12-14T04:47:34.714ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3b06fc13-84b8-4aa4-a155-e01503cb36d4 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26682024-12-14T04:47:34.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3b06fc13-84b8-4aa4-a155-e01503cb36d4 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26692024-12-14T04:47:34.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2271 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3b06fc13-84b8-4aa4-a155-e01503cb36d4 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
26702024-12-14T04:47:34.723ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-14T04:47:34.723654140Z }
26712024-12-14T04:47:34.723ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26722024-12-14T04:47:34.724ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26732024-12-14T04:47:34.732ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_needs_update = true
26742024-12-14T04:47:34.732ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26752024-12-14T04:47:34.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26762024-12-14T04:47:34.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26772024-12-14T04:47:34.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26782024-12-14T04:47:34.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26792024-12-14T04:47:34.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26802024-12-14T04:47:34.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26812024-12-14T04:47:34.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26822024-12-14T04:47:34.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26832024-12-14T04:47:34.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26842024-12-14T04:47:34.743ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26852024-12-14T04:47:34.743ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26862024-12-14T04:47:34.744ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26872024-12-14T04:47:34.747ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 79c3fe08-86fc-4863-afcc-4cee91996db1, lookup_type: ById(79c3fe08-86fc-4863-afcc-4cee91996db1) }, key: 29356bb4-514f-4536-9ee1-a2b09ca180f3, lookup_type: ById(29356bb4-514f-4536-9ee1-a2b09ca180f3) }, key: d9aefbd7-794b-4cb9-8265-f2cded56f0e3, lookup_type: ById(d9aefbd7-794b-4cb9-8265-f2cded56f0e3) } result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26882024-12-14T04:47:34.747ZINFOnexus (dropshot_internal): starting update saga for d9aefbd7-794b-4cb9-8265-f2cded56f0e3 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1583 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-14T04:47:34.723654140Z }
26892024-12-14T04:47:34.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 23824 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 response_code = 204 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26902024-12-14T04:47:34.747ZDEBGnexus (dropshot_internal): preparing instance-update saga for d9aefbd7-794b-4cb9-8265-f2cded56f0e3... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56b477e8-a22f-4cd3-91dd-c29d749af547 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26912024-12-14T04:47:34.747ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update
26922024-12-14T04:47:34.747ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"d9aefbd7-794b-4cb9-8265-f2cded56f0e3","lookup_type":{"ById":"d9aefbd7-794b-4cb9-8265-f2cded56f0e3"},"parent":{"key":"29356bb4-514f-4536-9ee1-a2b09ca180f3","lookup_type":{"ById":"29356bb4-514f-4536-9ee1-a2b09ca180f3"},"parent":{"key":"79c3fe08-86fc-4863-afcc-4cee91996db1","lookup_type":{"ById":"79c3fe08-86fc-4863-afcc-4cee91996db1"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
26932024-12-14T04:47:34.747ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update
26942024-12-14T04:47:34.748ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d245893b-e768-4b58-a064-7f5bfe896aab uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-14T04:47:34.723654140Z }
26952024-12-14T04:47:34.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d245893b-e768-4b58-a064-7f5bfe896aab roles = RoleSet { roles: {} } uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26962024-12-14T04:47:34.748ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d245893b-e768-4b58-a064-7f5bfe896aab resource = Database result = Ok(()) uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26972024-12-14T04:47:34.754ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6477 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d245893b-e768-4b58-a064-7f5bfe896aab response_code = 204 uri = /vmms/289c887e-2eb0-4375-9332-c2dea21acfce
26982024-12-14T04:47:34.787ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update
26992024-12-14T04:47:34.787ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27002024-12-14T04:47:34.787ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27012024-12-14T04:47:34.787ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27022024-12-14T04:47:34.794ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27032024-12-14T04:47:34.794ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27042024-12-14T04:47:34.799ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27052024-12-14T04:47:34.799ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27062024-12-14T04:47:34.804ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27072024-12-14T04:47:34.804ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("f4614af7-acf2-4926-a63b-dda6e0caf6a0")) node_id = 0 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27082024-12-14T04:47:34.808ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27092024-12-14T04:47:34.808ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27102024-12-14T04:47:34.815ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update
27112024-12-14T04:47:34.815ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27122024-12-14T04:47:34.815ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27132024-12-14T04:47:34.817ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27142024-12-14T04:47:34.817ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27152024-12-14T04:47:34.819ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27162024-12-14T04:47:34.819ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27172024-12-14T04:47:34.821ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27182024-12-14T04:47:34.821ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27192024-12-14T04:47:34.823ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27202024-12-14T04:47:34.823ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27212024-12-14T04:47:34.824ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27222024-12-14T04:47:34.824ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27232024-12-14T04:47:34.825ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27242024-12-14T04:47:34.829ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 79c3fe08-86fc-4863-afcc-4cee91996db1, lookup_type: ById(79c3fe08-86fc-4863-afcc-4cee91996db1) }, key: 29356bb4-514f-4536-9ee1-a2b09ca180f3, lookup_type: ById(29356bb4-514f-4536-9ee1-a2b09ca180f3) }, key: d9aefbd7-794b-4cb9-8265-f2cded56f0e3, lookup_type: ById(d9aefbd7-794b-4cb9-8265-f2cded56f0e3) } result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27252024-12-14T04:47:34.829ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance updater_id = f4614af7-acf2-4926-a63b-dda6e0caf6a0
27262024-12-14T04:47:34.829ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27272024-12-14T04:47:34.829ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27282024-12-14T04:47:34.846ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance updater_id = f4614af7-acf2-4926-a63b-dda6e0caf6a0
27292024-12-14T04:47:34.846ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27302024-12-14T04:47:34.847ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance
27312024-12-14T04:47:34.854ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 locked_gen = Generation(Generation(2)) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = LockInstance updater_id = f4614af7-acf2-4926-a63b-dda6e0caf6a0
27322024-12-14T04:47:34.854ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27332024-12-14T04:47:34.854ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("f4614af7-acf2-4926-a63b-dda6e0caf6a0")}) node_id = 1 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27342024-12-14T04:47:34.858ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27352024-12-14T04:47:34.858ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
27362024-12-14T04:47:34.862ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27372024-12-14T04:47:34.862ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27382024-12-14T04:47:34.863ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27392024-12-14T04:47:34.863ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27402024-12-14T04:47:34.865ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27412024-12-14T04:47:34.865ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27422024-12-14T04:47:34.866ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27432024-12-14T04:47:34.869ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 79c3fe08-86fc-4863-afcc-4cee91996db1, lookup_type: ById(79c3fe08-86fc-4863-afcc-4cee91996db1) }, key: 29356bb4-514f-4536-9ee1-a2b09ca180f3, lookup_type: ById(29356bb4-514f-4536-9ee1-a2b09ca180f3) }, key: d9aefbd7-794b-4cb9-8265-f2cded56f0e3, lookup_type: ById(d9aefbd7-794b-4cb9-8265-f2cded56f0e3) } result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27442024-12-14T04:47:34.869ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27452024-12-14T04:47:34.870ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27462024-12-14T04:47:34.875ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 289c887e-2eb0-4375-9332-c2dea21acfce, time_created: 2024-12-14T04:47:13.888832Z, time_deleted: None, instance_id: d9aefbd7-794b-4cb9-8265-f2cded56f0e3, sled_id: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2024-12-14T04:47:34.723654Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2024-12-14T04:47:11.713853Z, gen: Generation(Generation(3)), propolis_id: Some(289c887e-2eb0-4375-9332-c2dea21acfce), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(289c887e-2eb0-4375-9332-c2dea21acfce (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2024-12-14T04:47:34.875439635Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
27472024-12-14T04:47:34.877ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
27482024-12-14T04:47:34.877ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"d9aefbd7-794b-4cb9-8265-f2cded56f0e3","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"289c887e-2eb0-4375-9332-c2dea21acfce"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"d9aefbd7-794b-4cb9-8265-f2cded56f0e3","lookup_type":{"ById":"d9aefbd7-794b-4cb9-8265-f2cded56f0e3"},"parent":{"key":"29356bb4-514f-4536-9ee1-a2b09ca180f3","lookup_type":{"ById":"29356bb4-514f-4536-9ee1-a2b09ca180f3"},"parent":{"key":"79c3fe08-86fc-4863-afcc-4cee91996db1","lookup_type":{"ById":"79c3fe08-86fc-4863-afcc-4cee91996db1"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"f4614af7-acf2-4926-a63b-dda6e0caf6a0"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"29356bb4-514f-4536-9ee1-a2b09ca180f3","ram_diff":1073741824},"destroy_active_vmm":"289c887e-2eb0-4375-9332-c2dea21acfce","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2024-12-14T04:47:34.875439635Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27492024-12-14T04:47:34.877ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
27502024-12-14T04:47:34.909ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
27512024-12-14T04:47:34.909ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27522024-12-14T04:47:34.909ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27532024-12-14T04:47:34.909ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27542024-12-14T04:47:34.913ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27552024-12-14T04:47:34.913ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27562024-12-14T04:47:34.917ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27572024-12-14T04:47:34.917ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27582024-12-14T04:47:34.920ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27592024-12-14T04:47:34.920ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("a6d21f72-4913-4a5e-b401-5540843034fc")) node_id = 0 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27602024-12-14T04:47:34.923ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27612024-12-14T04:47:34.923ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27622024-12-14T04:47:34.931ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 parent_lock = UpdaterLock { updater_id: f4614af7-acf2-4926-a63b-dda6e0caf6a0, locked_gen: Generation(Generation(2)) } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
27632024-12-14T04:47:34.931ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = BecomeUpdater
27642024-12-14T04:47:34.932ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = BecomeUpdater
27652024-12-14T04:47:34.958ZDEBGnexus (SagaExecutor): inherited lock from f4614af7-acf2-4926-a63b-dda6e0caf6a0 to a6d21f72-4913-4a5e-b401-5540843034fc actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = f4614af7-acf2-4926-a63b-dda6e0caf6a0 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = BecomeUpdater updater_id = a6d21f72-4913-4a5e-b401-5540843034fc
27662024-12-14T04:47:34.958ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
27672024-12-14T04:47:34.958ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27682024-12-14T04:47:34.958ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("a6d21f72-4913-4a5e-b401-5540843034fc")}) node_id = 1 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27692024-12-14T04:47:34.962ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27702024-12-14T04:47:34.962ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27712024-12-14T04:47:34.968ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27722024-12-14T04:47:34.968ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27732024-12-14T04:47:34.971ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
27742024-12-14T04:47:34.971ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
27752024-12-14T04:47:34.975ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
27762024-12-14T04:47:34.980ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3
27772024-12-14T04:47:34.982ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UpdateNetworkConfig
27782024-12-14T04:47:34.983ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UpdateNetworkConfig
27792024-12-14T04:47:34.985ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UpdateNetworkConfig
27802024-12-14T04:47:34.985ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UpdateNetworkConfig
27812024-12-14T04:47:34.987ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
27822024-12-14T04:47:34.987ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UpdateNetworkConfig
27832024-12-14T04:47:34.987ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UpdateNetworkConfig
27842024-12-14T04:47:34.989ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
27852024-12-14T04:47:34.989ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
27862024-12-14T04:47:34.989ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
27872024-12-14T04:47:34.991ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
27882024-12-14T04:47:34.992ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
27892024-12-14T04:47:34.992ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
27902024-12-14T04:47:34.992ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
27912024-12-14T04:47:35.001ZDEBGnexus: notifying dendrite of updates instance_id = Some(d9aefbd7-794b-4cb9-8265-f2cded56f0e3 (instance)) switch = switch0
27922024-12-14T04:47:35.001ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1429, tv_nsec: 99086048 } })
27932024-12-14T04:47:35.001ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
27942024-12-14T04:47:35.032ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
27952024-12-14T04:47:35.032ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27962024-12-14T04:47:35.032ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9f00a03-4bfc-4a7a-9417-fec7a3f9c2b3", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:35 GMT"} })
27972024-12-14T04:47:35.032ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27982024-12-14T04:47:35.033ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27992024-12-14T04:47:35.062ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
28002024-12-14T04:47:35.062ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "6914d852-d844-4b50-ab45-031d6f1aa6ff", "content-length": "4", "date": "Sat, 14 Dec 2024 04:47:34 GMT"} })
28012024-12-14T04:47:35.063ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28022024-12-14T04:47:35.063ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28032024-12-14T04:47:35.068ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28042024-12-14T04:47:35.068ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28052024-12-14T04:47:35.074ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ReleaseVirtualProvisioning
28062024-12-14T04:47:35.074ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ReleaseVirtualProvisioning
28072024-12-14T04:47:35.088ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2024-12-14T04:47:35.076697Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 29356bb4-514f-4536-9ee1-a2b09ca180f3, time_modified: Some(2024-12-14T04:47:35.076697Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, time_modified: Some(2024-12-14T04:47:35.076697Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28082024-12-14T04:47:35.088ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28092024-12-14T04:47:35.088ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28102024-12-14T04:47:35.091ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28112024-12-14T04:47:35.091ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28122024-12-14T04:47:35.095ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28132024-12-14T04:47:35.097ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UnassignOximeterProducer
28142024-12-14T04:47:35.097ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UnassignOximeterProducer
28152024-12-14T04:47:35.102ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 producer_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28162024-12-14T04:47:35.102ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UnassignOximeterProducer
28172024-12-14T04:47:35.102ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = UnassignOximeterProducer
28182024-12-14T04:47:35.140ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28192024-12-14T04:47:35.142ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/d9aefbd7-794b-4cb9-8265-f2cded56f0e3
28202024-12-14T04:47:35.157ZDEBGnexus (SagaExecutor): client response oximeter-collector = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/d9aefbd7-794b-4cb9-8265-f2cded56f0e3", status: 204, headers: {"x-request-id": "0938046c-96bb-48cb-a142-87e862053b3d", "date": "Sat, 14 Dec 2024 04:47:35 GMT"} }) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28212024-12-14T04:47:35.157ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 producer_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28222024-12-14T04:47:35.159ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28232024-12-14T04:47:35.159ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28242024-12-14T04:47:35.163ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28252024-12-14T04:47:35.163ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28262024-12-14T04:47:35.166ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 lock = UpdaterLock { updater_id: a6d21f72-4913-4a5e-b401-5540843034fc, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2024-12-14T04:47:34.875439635Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28272024-12-14T04:47:35.166ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = CommitInstanceUpdates
28282024-12-14T04:47:35.167ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = CommitInstanceUpdates
28292024-12-14T04:47:35.204ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 new_runtime = InstanceRuntimeState { time_updated: 2024-12-14T04:47:34.875439635Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28302024-12-14T04:47:35.204ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28312024-12-14T04:47:35.204ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28322024-12-14T04:47:35.204ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Signaled
28332024-12-14T04:47:35.204ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Signaled
28342024-12-14T04:47:35.204ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28352024-12-14T04:47:35.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28362024-12-14T04:47:35.204ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28372024-12-14T04:47:35.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28382024-12-14T04:47:35.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28392024-12-14T04:47:35.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28402024-12-14T04:47:35.208ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28412024-12-14T04:47:35.208ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28422024-12-14T04:47:35.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28432024-12-14T04:47:35.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28442024-12-14T04:47:35.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28452024-12-14T04:47:35.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28462024-12-14T04:47:35.211ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28472024-12-14T04:47:35.211ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d9aefbd7-794b-4cb9-8265-f2cded56f0e3"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("289c887e-2eb0-4375-9332-c2dea21acfce")}) node_id = 7 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28482024-12-14T04:47:35.211ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28492024-12-14T04:47:35.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28502024-12-14T04:47:35.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28512024-12-14T04:47:35.215ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28522024-12-14T04:47:35.215ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28532024-12-14T04:47:35.243ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28542024-12-14T04:47:35.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28552024-12-14T04:47:35.245ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
28562024-12-14T04:47:35.245ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28572024-12-14T04:47:35.246ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "32d0ff6c-dce9-4038-b54c-506e6e4c2645", "content-length": "372", "date": "Sat, 14 Dec 2024 04:47:35 GMT"} })
28582024-12-14T04:47:35.246ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28592024-12-14T04:47:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28602024-12-14T04:47:35.249ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28612024-12-14T04:47:35.249ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28622024-12-14T04:47:35.253ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28632024-12-14T04:47:35.253ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28642024-12-14T04:47:35.261ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28652024-12-14T04:47:35.261ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ReleaseSledResources
28662024-12-14T04:47:35.262ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ReleaseSledResources
28672024-12-14T04:47:35.273ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28682024-12-14T04:47:35.273ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28692024-12-14T04:47:35.277ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28702024-12-14T04:47:35.277ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28712024-12-14T04:47:35.280ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 propolis_id = 289c887e-2eb0-4375-9332-c2dea21acfce saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
28722024-12-14T04:47:35.280ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = MarkVmmDeleted
28732024-12-14T04:47:35.280ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = MarkVmmDeleted
28742024-12-14T04:47:35.293ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28752024-12-14T04:47:35.293ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28762024-12-14T04:47:35.296ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {19e22398-f58e-423e-b5da-53936351a2a8: {10.151.1.176: {7ca852b6-7025-4b88-a386-e1170276dc34}, 10.151.1.175: {7ca852b6-7025-4b88-a386-e1170276dc34}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
28772024-12-14T04:47:35.296ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
28782024-12-14T04:47:35.296ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28792024-12-14T04:47:35.296ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28802024-12-14T04:47:35.296ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "17552b99-9fb7-4527-b91f-75ff8b2d092f", "date": "Sat, 14 Dec 2024 04:47:35 GMT"} })
28812024-12-14T04:47:35.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28822024-12-14T04:47:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28832024-12-14T04:47:35.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28842024-12-14T04:47:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28852024-12-14T04:47:35.300ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28862024-12-14T04:47:35.300ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28872024-12-14T04:47:35.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28882024-12-14T04:47:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28892024-12-14T04:47:35.304ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
28902024-12-14T04:47:35.304ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
28912024-12-14T04:47:35.306ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
28922024-12-14T04:47:35.306ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28932024-12-14T04:47:35.307ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a39780bc-a5a6-4bff-8028-7f102ea8ecdb", "date": "Sat, 14 Dec 2024 04:47:35 GMT"} })
28942024-12-14T04:47:35.307ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
28952024-12-14T04:47:35.307ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
28962024-12-14T04:47:35.308ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
28972024-12-14T04:47:35.308ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 103.98079ms iteration = 27 status = {}
28982024-12-14T04:47:35.308ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
28992024-12-14T04:47:35.337ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
29002024-12-14T04:47:35.337ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
29012024-12-14T04:47:35.338ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29022024-12-14T04:47:35.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29032024-12-14T04:47:35.366ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
29042024-12-14T04:47:35.367ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29052024-12-14T04:47:35.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29062024-12-14T04:47:35.369ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29072024-12-14T04:47:35.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29082024-12-14T04:47:35.370ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 79c3fe08-86fc-4863-afcc-4cee91996db1, lookup_type: ById(79c3fe08-86fc-4863-afcc-4cee91996db1) }, key: 29356bb4-514f-4536-9ee1-a2b09ca180f3, lookup_type: ById(29356bb4-514f-4536-9ee1-a2b09ca180f3) }, key: d9aefbd7-794b-4cb9-8265-f2cded56f0e3, lookup_type: ById(d9aefbd7-794b-4cb9-8265-f2cded56f0e3) } result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
29092024-12-14T04:47:35.370ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
29102024-12-14T04:47:35.370ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29112024-12-14T04:47:35.371ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_node = ChainSuccessorSaga
29122024-12-14T04:47:35.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29132024-12-14T04:47:35.372ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29142024-12-14T04:47:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29152024-12-14T04:47:35.375ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 karmic_state = WillReincarnate needs_reincarnation = false saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update
29162024-12-14T04:47:35.376ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29172024-12-14T04:47:35.376ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
29182024-12-14T04:47:35.379ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29192024-12-14T04:47:35.379ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
29202024-12-14T04:47:35.382ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29212024-12-14T04:47:35.382ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
29222024-12-14T04:47:35.385ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29232024-12-14T04:47:35.385ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa
29242024-12-14T04:47:35.430ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29252024-12-14T04:47:35.430ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa saga_name = instance-update saga_result = SagaResult { saga_id: fecf685f-09b4-4d06-9625-34e8654ba4fa, saga_log: SagaLog { saga_id: fecf685f-09b4-4d06-9625-34e8654ba4fa, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("a6d21f72-4913-4a5e-b401-5540843034fc")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("a6d21f72-4913-4a5e-b401-5540843034fc")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("d9aefbd7-794b-4cb9-8265-f2cded56f0e3"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("289c887e-2eb0-4375-9332-c2dea21acfce")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("d9aefbd7-794b-4cb9-8265-f2cded56f0e3"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("289c887e-2eb0-4375-9332-c2dea21acfce")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("a6d21f72-4913-4a5e-b401-5540843034fc"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("a6d21f72-4913-4a5e-b401-5540843034fc")}} }) }
29262024-12-14T04:47:35.430ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = fecf685f-09b4-4d06-9625-34e8654ba4fa instance_id = d9aefbd7-794b-4cb9-8265-f2cded56f0e3 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update
29272024-12-14T04:47:35.430ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29282024-12-14T04:47:35.430ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
29292024-12-14T04:47:35.434ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29302024-12-14T04:47:35.434ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
29312024-12-14T04:47:35.438ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29322024-12-14T04:47:35.438ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
29332024-12-14T04:47:35.441ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29342024-12-14T04:47:35.441ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = faede768-251d-4ab5-9ccf-37e868e2c078
29352024-12-14T04:47:35.471ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
29362024-12-14T04:47:35.471ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "91494847-903f-4d22-9a50-2dc1a74acb48", "content-length": "691", "date": "Sat, 14 Dec 2024 04:47:35 GMT"} })
29372024-12-14T04:47:35.471ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
29382024-12-14T04:47:35.471ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
29392024-12-14T04:47:35.471ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
29402024-12-14T04:47:35.473ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update sec_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
29412024-12-14T04:47:35.473ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = faede768-251d-4ab5-9ccf-37e868e2c078 saga_name = start-instance-update saga_result = SagaResult { saga_id: faede768-251d-4ab5-9ccf-37e868e2c078, saga_log: SagaLog { saga_id: faede768-251d-4ab5-9ccf-37e868e2c078, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("f4614af7-acf2-4926-a63b-dda6e0caf6a0")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("f4614af7-acf2-4926-a63b-dda6e0caf6a0")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("f4614af7-acf2-4926-a63b-dda6e0caf6a0"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("f4614af7-acf2-4926-a63b-dda6e0caf6a0")}} }) }
29422024-12-14T04:47:35.474ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 269.440501ms iteration = 26 status = {}
29432024-12-14T04:47:35.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca240c40-7e54-466d-805c-dfc22313b7cd roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
29442024-12-14T04:47:35.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca240c40-7e54-466d-805c-dfc22313b7cd resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
29452024-12-14T04:47:35.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2849 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca240c40-7e54-466d-805c-dfc22313b7cd response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
29462024-12-14T04:47:35.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 4abd7466-b1c2-40f4-84d5-8bd3448551eb roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
29472024-12-14T04:47:35.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 4abd7466-b1c2-40f4-84d5-8bd3448551eb resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
29482024-12-14T04:47:35.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3563 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 4abd7466-b1c2-40f4-84d5-8bd3448551eb response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
29492024-12-14T04:47:36.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 61a87aff-0b0d-43db-b3b6-8e5b3c4efa2f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
29502024-12-14T04:47:36.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 61a87aff-0b0d-43db-b3b6-8e5b3c4efa2f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
29512024-12-14T04:47:36.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1875 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 61a87aff-0b0d-43db-b3b6-8e5b3c4efa2f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
29522024-12-14T04:47:36.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 9531231f-c866-40e9-aca5-0f0255b40392 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
29532024-12-14T04:47:36.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 9531231f-c866-40e9-aca5-0f0255b40392 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
29542024-12-14T04:47:36.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2826 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 9531231f-c866-40e9-aca5-0f0255b40392 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
29552024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
29562024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
29572024-12-14T04:47:36.881ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
29582024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
29592024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29602024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
29612024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29622024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
29632024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
29642024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29652024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29662024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
29672024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29682024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
29692024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
29702024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
29712024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29722024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
29732024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29742024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
29752024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29762024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
29772024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
29782024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29792024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29802024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
29812024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
29822024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29832024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
29842024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
29852024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29862024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
29872024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29882024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29892024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29902024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29912024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
29922024-12-14T04:47:36.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29932024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29942024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29952024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
29962024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29972024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29982024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29992024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30002024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
30012024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
30022024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30032024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30042024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30052024-12-14T04:47:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30062024-12-14T04:47:36.884ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30072024-12-14T04:47:36.884ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30082024-12-14T04:47:36.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1430, tv_nsec: 983734105 } })
30092024-12-14T04:47:36.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30102024-12-14T04:47:36.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30112024-12-14T04:47:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30122024-12-14T04:47:36.886ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
30132024-12-14T04:47:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30142024-12-14T04:47:36.887ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30152024-12-14T04:47:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30162024-12-14T04:47:36.890ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1430, tv_nsec: 988134219 } })
30172024-12-14T04:47:36.891ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
30182024-12-14T04:47:36.898ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.129547ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
30192024-12-14T04:47:36.898ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1430, tv_nsec: 996239670 } })
30202024-12-14T04:47:36.899ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
30212024-12-14T04:47:36.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 40.374312ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
30222024-12-14T04:47:36.927ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30232024-12-14T04:47:36.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30242024-12-14T04:47:36.928ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30252024-12-14T04:47:36.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30262024-12-14T04:47:36.929ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30272024-12-14T04:47:36.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30282024-12-14T04:47:36.931ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 49.513621ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
30292024-12-14T04:47:36.931ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30302024-12-14T04:47:36.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30312024-12-14T04:47:36.933ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30322024-12-14T04:47:36.934ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30332024-12-14T04:47:36.934ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
30342024-12-14T04:47:36.934ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30352024-12-14T04:47:36.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30362024-12-14T04:47:36.936ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30372024-12-14T04:47:36.936ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30382024-12-14T04:47:36.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30392024-12-14T04:47:36.938ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 56.590934ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
30402024-12-14T04:47:36.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30412024-12-14T04:47:36.939ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30422024-12-14T04:47:36.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30432024-12-14T04:47:36.939ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30442024-12-14T04:47:36.941ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30452024-12-14T04:47:36.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30462024-12-14T04:47:36.941ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30472024-12-14T04:47:36.941ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30482024-12-14T04:47:36.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30492024-12-14T04:47:36.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30502024-12-14T04:47:36.942ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30512024-12-14T04:47:36.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30522024-12-14T04:47:36.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30532024-12-14T04:47:36.944ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30542024-12-14T04:47:36.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30552024-12-14T04:47:36.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30562024-12-14T04:47:36.944ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30572024-12-14T04:47:36.946ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30582024-12-14T04:47:36.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30592024-12-14T04:47:36.946ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30602024-12-14T04:47:36.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30612024-12-14T04:47:36.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30622024-12-14T04:47:36.948ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30632024-12-14T04:47:36.948ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30642024-12-14T04:47:36.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30652024-12-14T04:47:36.949ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30662024-12-14T04:47:36.949ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30672024-12-14T04:47:36.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30682024-12-14T04:47:36.970ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 89.250275ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
30692024-12-14T04:47:36.971ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 90.292543ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30702024-12-14T04:47:36.971ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30712024-12-14T04:47:36.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30722024-12-14T04:47:36.972ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30732024-12-14T04:47:36.972ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
30742024-12-14T04:47:36.972ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30752024-12-14T04:47:36.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30762024-12-14T04:47:36.973ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30772024-12-14T04:47:36.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30782024-12-14T04:47:36.975ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
30792024-12-14T04:47:36.975ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30802024-12-14T04:47:36.975ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9379272-17c4-41b1-b555-4d477c15fec5", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
30812024-12-14T04:47:36.975ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30822024-12-14T04:47:36.976ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30832024-12-14T04:47:36.977ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30842024-12-14T04:47:36.977ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 96.299519ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30852024-12-14T04:47:36.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
30862024-12-14T04:47:36.978ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30872024-12-14T04:47:36.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30882024-12-14T04:47:36.980ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 99.239766ms iteration = 26 status = 0
30892024-12-14T04:47:36.985ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
30902024-12-14T04:47:36.985ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.436969ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
30912024-12-14T04:47:36.985ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
30922024-12-14T04:47:36.985ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30932024-12-14T04:47:36.985ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "16d0453c-2a32-4dd1-a480-5a7044c5566a", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
30942024-12-14T04:47:36.985ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30952024-12-14T04:47:36.987ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30962024-12-14T04:47:36.990ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
30972024-12-14T04:47:36.990ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30982024-12-14T04:47:36.990ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dad70947-78e8-4e33-8a4e-6a0727431944", "content-length": "26", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
30992024-12-14T04:47:36.990ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31002024-12-14T04:47:36.993ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
31012024-12-14T04:47:36.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31022024-12-14T04:47:36.996ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
31032024-12-14T04:47:36.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31042024-12-14T04:47:36.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31052024-12-14T04:47:37.010ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
31062024-12-14T04:47:37.010ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 128.327014ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
31072024-12-14T04:47:37.015ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31082024-12-14T04:47:37.015ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
31092024-12-14T04:47:37.015ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 134.288649ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
31102024-12-14T04:47:37.023ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31112024-12-14T04:47:37.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31122024-12-14T04:47:37.058ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 177.30211ms iteration = 26 status = {}
31132024-12-14T04:47:37.060ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31142024-12-14T04:47:37.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31152024-12-14T04:47:37.062ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31162024-12-14T04:47:37.064ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 182.66046ms iteration = 30 status = {"physical_disks_added":0}
31172024-12-14T04:47:37.064ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31182024-12-14T04:47:37.064ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31192024-12-14T04:47:37.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31202024-12-14T04:47:37.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31212024-12-14T04:47:37.067ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31222024-12-14T04:47:37.067ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31232024-12-14T04:47:37.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31242024-12-14T04:47:37.071ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
31252024-12-14T04:47:37.071ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc2489fc-9dab-4d89-bd0b-918272d1f0a9", "content-length": "2", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
31262024-12-14T04:47:37.073ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31272024-12-14T04:47:37.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31282024-12-14T04:47:37.114ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
31292024-12-14T04:47:37.114ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31302024-12-14T04:47:37.114ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "58239711-1b22-4753-8bbe-dd35ee6fdc29", "content-length": "372", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
31312024-12-14T04:47:37.114ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31322024-12-14T04:47:37.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31332024-12-14T04:47:37.116ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 235.380233ms iteration = 26 status = null
31342024-12-14T04:47:37.157ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
31352024-12-14T04:47:37.157ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
31362024-12-14T04:47:37.157ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "25de8117-a688-49bb-a1aa-cd202765c514", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
31372024-12-14T04:47:37.157ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31382024-12-14T04:47:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31392024-12-14T04:47:37.159ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31402024-12-14T04:47:37.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31412024-12-14T04:47:37.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31422024-12-14T04:47:37.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31432024-12-14T04:47:37.162ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
31442024-12-14T04:47:37.162ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31452024-12-14T04:47:37.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31462024-12-14T04:47:37.162ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31472024-12-14T04:47:37.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31482024-12-14T04:47:37.166ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
31492024-12-14T04:47:37.166ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31502024-12-14T04:47:37.166ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "05de92e4-9d05-4709-8e3a-a82d2cd2c55f", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
31512024-12-14T04:47:37.168ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 286.69025ms iteration = 28 status = {}
31522024-12-14T04:47:37.171ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
31532024-12-14T04:47:37.171ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8524b08f-da5f-4517-ad05-644453b83f4a", "content-length": "134", "date": "Sat, 14 Dec 2024 04:47:37 GMT"} })
31542024-12-14T04:47:37.171ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
31552024-12-14T04:47:37.171ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
31562024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
31572024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
31582024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31592024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31602024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31612024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
31622024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31632024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
31642024-12-14T04:47:37.172ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
31652024-12-14T04:47:37.172ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31662024-12-14T04:47:37.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31672024-12-14T04:47:37.174ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
31682024-12-14T04:47:37.174ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8397139-8082-426d-8dd0-3bd19660f42b", "content-length": "2", "date": "Sat, 14 Dec 2024 04:47:37 GMT"} })
31692024-12-14T04:47:37.174ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
31702024-12-14T04:47:37.174ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e3e5f8a-fde8-43fe-a876-680f9a809aa4", "content-length": "52", "date": "Sat, 14 Dec 2024 04:47:37 GMT"} })
31712024-12-14T04:47:37.174ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
31722024-12-14T04:47:37.174ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
31732024-12-14T04:47:37.202ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
31742024-12-14T04:47:37.274ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0ae529df-6541-4b0f-b02c-8118dd9b72e2", "date": "Sat, 14 Dec 2024 04:47:37 GMT"} })
31752024-12-14T04:47:37.274ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
31762024-12-14T04:47:37.274ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31772024-12-14T04:47:37.274ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9569ea5-fe4f-41db-8d10-70237a70eb23", "content-length": "682", "date": "Sat, 14 Dec 2024 04:47:36 GMT"} })
31782024-12-14T04:47:37.274ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "eefacaa8-243c-4690-a266-df32446dc39f", "content-length": "691", "date": "Sat, 14 Dec 2024 04:47:37 GMT"} })
31792024-12-14T04:47:37.274ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31802024-12-14T04:47:37.274ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
31812024-12-14T04:47:37.274ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
31822024-12-14T04:47:37.274ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
31832024-12-14T04:47:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31842024-12-14T04:47:37.276ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 394.736266ms iteration = 27 status = {}
31852024-12-14T04:47:37.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31862024-12-14T04:47:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31872024-12-14T04:47:37.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31882024-12-14T04:47:37.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31892024-12-14T04:47:37.283ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31902024-12-14T04:47:37.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31912024-12-14T04:47:37.291ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 409.88078ms iteration = 26 status = {}
31922024-12-14T04:47:37.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 747c5e08-8202-497a-8a7b-85f47927a464 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31932024-12-14T04:47:37.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 747c5e08-8202-497a-8a7b-85f47927a464 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31942024-12-14T04:47:37.784ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2455 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 747c5e08-8202-497a-8a7b-85f47927a464 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31952024-12-14T04:47:38.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 64a8b6d3-b678-481e-800f-a94b6a1aea2b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31962024-12-14T04:47:38.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 64a8b6d3-b678-481e-800f-a94b6a1aea2b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31972024-12-14T04:47:38.808ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2784 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 64a8b6d3-b678-481e-800f-a94b6a1aea2b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31982024-12-14T04:47:39.829ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7a68ee51-b3c9-491d-807c-70bf89c46298 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
31992024-12-14T04:47:39.829ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7a68ee51-b3c9-491d-807c-70bf89c46298 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32002024-12-14T04:47:39.831ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2500 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7a68ee51-b3c9-491d-807c-70bf89c46298 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32012024-12-14T04:47:40.852ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a2ad248b-0bb9-4ed8-8693-17876264cd11 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32022024-12-14T04:47:40.852ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a2ad248b-0bb9-4ed8-8693-17876264cd11 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32032024-12-14T04:47:40.854ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2740 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a2ad248b-0bb9-4ed8-8693-17876264cd11 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32042024-12-14T04:47:41.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 30630281-46f9-4536-a498-165fb6ad8526 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32052024-12-14T04:47:41.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 30630281-46f9-4536-a498-165fb6ad8526 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32062024-12-14T04:47:41.877ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2086 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 30630281-46f9-4536-a498-165fb6ad8526 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32072024-12-14T04:47:42.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1504 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 9e53baa2-1c20-48bb-a30b-701157b77554 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
32082024-12-14T04:47:42.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3824651c-6ba8-4e21-81ec-bc360c2f67a9 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32092024-12-14T04:47:42.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3824651c-6ba8-4e21-81ec-bc360c2f67a9 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32102024-12-14T04:47:42.899ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2503 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3824651c-6ba8-4e21-81ec-bc360c2f67a9 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32112024-12-14T04:47:43.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1c2c2b4e-1c0f-4797-9b7d-92b8352972f0 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32122024-12-14T04:47:43.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1c2c2b4e-1c0f-4797-9b7d-92b8352972f0 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32132024-12-14T04:47:43.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2510 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1c2c2b4e-1c0f-4797-9b7d-92b8352972f0 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32142024-12-14T04:47:44.942ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74a72369-5fcb-41ea-9d39-bb341dc38655 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32152024-12-14T04:47:44.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74a72369-5fcb-41ea-9d39-bb341dc38655 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32162024-12-14T04:47:44.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2334 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74a72369-5fcb-41ea-9d39-bb341dc38655 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32172024-12-14T04:47:45.964ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = dce787b5-bf5f-43ec-b477-7a1bf3d8e877 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32182024-12-14T04:47:45.964ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = dce787b5-bf5f-43ec-b477-7a1bf3d8e877 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32192024-12-14T04:47:45.966ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2461 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = dce787b5-bf5f-43ec-b477-7a1bf3d8e877 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32202024-12-14T04:47:46.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
32212024-12-14T04:47:46.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32222024-12-14T04:47:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32232024-12-14T04:47:46.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
32242024-12-14T04:47:46.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32252024-12-14T04:47:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32262024-12-14T04:47:46.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32272024-12-14T04:47:46.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32282024-12-14T04:47:46.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32292024-12-14T04:47:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32302024-12-14T04:47:46.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32312024-12-14T04:47:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32322024-12-14T04:47:46.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32332024-12-14T04:47:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32342024-12-14T04:47:46.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32352024-12-14T04:47:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
32362024-12-14T04:47:46.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32372024-12-14T04:47:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32382024-12-14T04:47:46.904ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
32392024-12-14T04:47:46.904ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.788497ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
32402024-12-14T04:47:46.912ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.564496ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32412024-12-14T04:47:46.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1adb6f8-e008-4cdf-863c-df2350e60d2a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32422024-12-14T04:47:46.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1adb6f8-e008-4cdf-863c-df2350e60d2a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32432024-12-14T04:47:46.987ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1715 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1adb6f8-e008-4cdf-863c-df2350e60d2a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32442024-12-14T04:47:48.008ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e031c357-1a59-4cea-8aac-9997f4c7277f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32452024-12-14T04:47:48.008ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e031c357-1a59-4cea-8aac-9997f4c7277f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32462024-12-14T04:47:48.009ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1981 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e031c357-1a59-4cea-8aac-9997f4c7277f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32472024-12-14T04:47:49.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 076bbbb5-3b36-4a36-ae0f-cf9a4d882c01 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32482024-12-14T04:47:49.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 076bbbb5-3b36-4a36-ae0f-cf9a4d882c01 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32492024-12-14T04:47:49.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2445 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 076bbbb5-3b36-4a36-ae0f-cf9a4d882c01 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32502024-12-14T04:47:50.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ea7594d6-4073-44d4-b17e-4b9cf732663e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32512024-12-14T04:47:50.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ea7594d6-4073-44d4-b17e-4b9cf732663e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32522024-12-14T04:47:50.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2254 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ea7594d6-4073-44d4-b17e-4b9cf732663e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32532024-12-14T04:47:50.270ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:65473
32542024-12-14T04:47:50.271ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:65473 req_id = 47b45946-1faf-4ed4-8292-a94e63c3ce53 roles = RoleSet { roles: {} } uri = /metrics/producers
32552024-12-14T04:47:50.271ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:65473 req_id = 47b45946-1faf-4ed4-8292-a94e63c3ce53 resource = Database result = Ok(()) uri = /metrics/producers
32562024-12-14T04:47:50.307ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 5f4909e4-dcea-4db6-99c3-a64609b9f774
32572024-12-14T04:47:50.307ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36856 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:65473 req_id = 47b45946-1faf-4ed4-8292-a94e63c3ce53 response_code = 201 uri = /metrics/producers
32582024-12-14T04:47:51.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 48f44da4-8712-45b7-95b4-093ce54f1d94 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32592024-12-14T04:47:51.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 48f44da4-8712-45b7-95b4-093ce54f1d94 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32602024-12-14T04:47:51.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2437 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 48f44da4-8712-45b7-95b4-093ce54f1d94 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32612024-12-14T04:47:52.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8f69d61-0051-4578-ac6a-0fc9e8b9f1ee roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32622024-12-14T04:47:52.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8f69d61-0051-4578-ac6a-0fc9e8b9f1ee resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32632024-12-14T04:47:52.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2216 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8f69d61-0051-4578-ac6a-0fc9e8b9f1ee response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32642024-12-14T04:47:52.380ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
32652024-12-14T04:47:52.380ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
32662024-12-14T04:47:52.410ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
32672024-12-14T04:47:52.411ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:62052
32682024-12-14T04:47:52.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62052 req_id = 9ba09b55-4cb7-4a39-b928-21e7aa86449b roles = RoleSet { roles: {} } uri = /metrics/producers
32692024-12-14T04:47:52.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62052 req_id = 9ba09b55-4cb7-4a39-b928-21e7aa86449b resource = Database result = Ok(()) uri = /metrics/producers
32702024-12-14T04:47:52.448ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
32712024-12-14T04:47:52.448ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37577 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62052 req_id = 9ba09b55-4cb7-4a39-b928-21e7aa86449b response_code = 201 uri = /metrics/producers
32722024-12-14T04:47:52.448ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "9ba09b55-4cb7-4a39-b928-21e7aa86449b", "content-length": "41", "date": "Sat, 14 Dec 2024 04:47:52 GMT"} })
32732024-12-14T04:47:52.450ZDEBGnexus (producer-registration-task): registered with nexus successfully
32742024-12-14T04:47:52.450ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
32752024-12-14T04:47:52.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1385 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 2134c535-aca4-4fe7-9f94-68c02949e26f response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
32762024-12-14T04:47:53.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 074c642c-b6ea-43c2-bf70-9765a66062bc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32772024-12-14T04:47:53.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 074c642c-b6ea-43c2-bf70-9765a66062bc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32782024-12-14T04:47:53.119ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2506 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 074c642c-b6ea-43c2-bf70-9765a66062bc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32792024-12-14T04:47:54.139ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e98422bc-64f5-40d7-a05b-dd1ce507baa5 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32802024-12-14T04:47:54.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e98422bc-64f5-40d7-a05b-dd1ce507baa5 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32812024-12-14T04:47:54.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2008 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e98422bc-64f5-40d7-a05b-dd1ce507baa5 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32822024-12-14T04:47:54.214ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:45790
32832024-12-14T04:47:54.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45790 req_id = 85004038-d932-4deb-864f-95a974a05949 roles = RoleSet { roles: {} } uri = /metrics/producers
32842024-12-14T04:47:54.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45790 req_id = 85004038-d932-4deb-864f-95a974a05949 resource = Database result = Ok(()) uri = /metrics/producers
32852024-12-14T04:47:54.257ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = b1576088-d04d-48cf-a285-75e4628d77d5
32862024-12-14T04:47:54.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43023 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45790 req_id = 85004038-d932-4deb-864f-95a974a05949 response_code = 201 uri = /metrics/producers
32872024-12-14T04:47:55.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0992a305-9ea0-4ed2-972e-e9abf4cafe6c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32882024-12-14T04:47:55.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0992a305-9ea0-4ed2-972e-e9abf4cafe6c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32892024-12-14T04:47:55.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2425 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0992a305-9ea0-4ed2-972e-e9abf4cafe6c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32902024-12-14T04:47:56.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = be15ad9e-b9fa-49cc-b964-401f58be8eee roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32912024-12-14T04:47:56.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = be15ad9e-b9fa-49cc-b964-401f58be8eee resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32922024-12-14T04:47:56.185ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2127 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = be15ad9e-b9fa-49cc-b964-401f58be8eee response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
32932024-12-14T04:47:56.533ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:39965
32942024-12-14T04:47:56.533ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39965 req_id = b5ed144e-33f9-4db2-b2cb-c31a9847c24b roles = RoleSet { roles: {} } uri = /metrics/producers
32952024-12-14T04:47:56.533ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39965 req_id = b5ed144e-33f9-4db2-b2cb-c31a9847c24b resource = Database result = Ok(()) uri = /metrics/producers
32962024-12-14T04:47:56.571ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 83153bbf-d864-44c8-b8a5-770b4235fe7a
32972024-12-14T04:47:56.571ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38722 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39965 req_id = b5ed144e-33f9-4db2-b2cb-c31a9847c24b response_code = 201 uri = /metrics/producers
32982024-12-14T04:47:56.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
32992024-12-14T04:47:56.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33002024-12-14T04:47:56.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33012024-12-14T04:47:56.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
33022024-12-14T04:47:56.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33032024-12-14T04:47:56.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33042024-12-14T04:47:56.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33052024-12-14T04:47:56.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33062024-12-14T04:47:56.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33072024-12-14T04:47:56.885ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.282461ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33082024-12-14T04:47:56.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33092024-12-14T04:47:56.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33102024-12-14T04:47:56.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33112024-12-14T04:47:56.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33122024-12-14T04:47:56.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33132024-12-14T04:47:56.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33142024-12-14T04:47:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
33152024-12-14T04:47:56.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33162024-12-14T04:47:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33172024-12-14T04:47:56.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
33182024-12-14T04:47:56.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.979561ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
33192024-12-14T04:47:57.204ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1635d590-0df1-4ef9-8b42-3e0b79204b61 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33202024-12-14T04:47:57.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1635d590-0df1-4ef9-8b42-3e0b79204b61 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33212024-12-14T04:47:57.206ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2023 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1635d590-0df1-4ef9-8b42-3e0b79204b61 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33222024-12-14T04:47:58.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5e7f9557-b442-4fa2-96c0-1a262883ac8a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33232024-12-14T04:47:58.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5e7f9557-b442-4fa2-96c0-1a262883ac8a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33242024-12-14T04:47:58.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2195 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5e7f9557-b442-4fa2-96c0-1a262883ac8a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33252024-12-14T04:47:59.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 228c9500-090f-45f9-abe0-9c0703a55b0a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33262024-12-14T04:47:59.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 228c9500-090f-45f9-abe0-9c0703a55b0a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33272024-12-14T04:47:59.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2674 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 228c9500-090f-45f9-abe0-9c0703a55b0a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33282024-12-14T04:48:00.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e03c9fc7-b98b-484f-979c-e4e5425915ab roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33292024-12-14T04:48:00.271ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e03c9fc7-b98b-484f-979c-e4e5425915ab resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33302024-12-14T04:48:00.273ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2841 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e03c9fc7-b98b-484f-979c-e4e5425915ab response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33312024-12-14T04:48:01.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74809322-9db0-42f0-9ed0-d0781055f789 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33322024-12-14T04:48:01.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74809322-9db0-42f0-9ed0-d0781055f789 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33332024-12-14T04:48:01.295ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1876 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 74809322-9db0-42f0-9ed0-d0781055f789 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33342024-12-14T04:48:02.314ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f260ed2d-7880-442f-9634-e92e30e29e8e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33352024-12-14T04:48:02.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f260ed2d-7880-442f-9634-e92e30e29e8e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33362024-12-14T04:48:02.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2691 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f260ed2d-7880-442f-9634-e92e30e29e8e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33372024-12-14T04:48:02.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 57533e27-baa1-4e37-b505-68bec1114b90 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
33382024-12-14T04:48:02.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 57533e27-baa1-4e37-b505-68bec1114b90 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
33392024-12-14T04:48:02.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3524 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 57533e27-baa1-4e37-b505-68bec1114b90 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
33402024-12-14T04:48:02.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 3c57488a-2d5f-4b41-8ce9-59a4b79552a2 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
33412024-12-14T04:48:02.774ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 3c57488a-2d5f-4b41-8ce9-59a4b79552a2 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
33422024-12-14T04:48:02.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2796 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 3c57488a-2d5f-4b41-8ce9-59a4b79552a2 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
33432024-12-14T04:48:02.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1316 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 53c3fa19-cfee-4273-8acb-0f8d647aa40c response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
33442024-12-14T04:48:03.336ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34e9ea3d-10de-481a-aa12-f12b16421ce1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33452024-12-14T04:48:03.336ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34e9ea3d-10de-481a-aa12-f12b16421ce1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33462024-12-14T04:48:03.338ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2283 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34e9ea3d-10de-481a-aa12-f12b16421ce1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33472024-12-14T04:48:04.360ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3917ba04-a759-4134-b816-57724abc748d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33482024-12-14T04:48:04.360ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3917ba04-a759-4134-b816-57724abc748d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33492024-12-14T04:48:04.362ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2153 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3917ba04-a759-4134-b816-57724abc748d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33502024-12-14T04:48:05.382ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d410d26f-f928-4602-acb9-0af01f55dff2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33512024-12-14T04:48:05.382ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d410d26f-f928-4602-acb9-0af01f55dff2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33522024-12-14T04:48:05.384ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2643 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d410d26f-f928-4602-acb9-0af01f55dff2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33532024-12-14T04:48:06.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 901df4bf-d9f7-4098-ad4e-53abefa92410 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33542024-12-14T04:48:06.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 901df4bf-d9f7-4098-ad4e-53abefa92410 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33552024-12-14T04:48:06.408ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2440 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 901df4bf-d9f7-4098-ad4e-53abefa92410 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
33562024-12-14T04:48:06.822ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 99896e9e-d225-43e0-8b34-f2a4e4165233 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
33572024-12-14T04:48:06.822ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 99896e9e-d225-43e0-8b34-f2a4e4165233 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
33582024-12-14T04:48:06.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3243 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 99896e9e-d225-43e0-8b34-f2a4e4165233 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
33592024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
33602024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
33612024-12-14T04:48:06.881ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
33622024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
33632024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
33642024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
33652024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
33662024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33672024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
33682024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
33692024-12-14T04:48:06.881ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
33702024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
33712024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
33722024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33732024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
33742024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33752024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
33762024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
33772024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
33782024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
33792024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
33802024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33812024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33822024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
33832024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
33842024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33852024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
33862024-12-14T04:48:06.881ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832
33872024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.131\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832"}
33882024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
33892024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
33902024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33912024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
33922024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
33932024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
33942024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
33952024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
33962024-12-14T04:48:06.881ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 04:38:06.881648730 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
33972024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
33982024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33992024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
34002024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
34012024-12-14T04:48:06.881ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
34022024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34032024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
34042024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34052024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
34062024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34072024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
34082024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34092024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
34102024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34112024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
34122024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34132024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
34142024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34152024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
34162024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
34172024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
34182024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
34192024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
34202024-12-14T04:48:06.882ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ba1572d-df24-4bd2-be03-9a99edc867d8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1460, tv_nsec: 979539606 } })
34212024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34222024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34232024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34242024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34252024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
34262024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34272024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34282024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
34292024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
34302024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
34312024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34322024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34332024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
34342024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
34352024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34362024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
34372024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34382024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34392024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
34402024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34412024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34422024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34432024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34442024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34452024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
34462024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
34472024-12-14T04:48:06.882ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.123439ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
34482024-12-14T04:48:06.882ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2224041a-a9fc-46d2-b9ed-ef98dc01f696.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("229c9d57-e318-46f6-8df9-9f602a126f12.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1460, tv_nsec: 980004207 } })
34492024-12-14T04:48:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
34502024-12-14T04:48:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 839.202\u{b5}s iteration = 15 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
34512024-12-14T04:48:06.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34522024-12-14T04:48:06.884ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34532024-12-14T04:48:06.885ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.104778ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
34542024-12-14T04:48:06.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34552024-12-14T04:48:06.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34562024-12-14T04:48:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34572024-12-14T04:48:06.886ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
34582024-12-14T04:48:06.886ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
34592024-12-14T04:48:06.886ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 4.664481ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
34602024-12-14T04:48:06.886ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.800855ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
34612024-12-14T04:48:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
34622024-12-14T04:48:06.886ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
34632024-12-14T04:48:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
34642024-12-14T04:48:06.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34652024-12-14T04:48:06.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34662024-12-14T04:48:06.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34672024-12-14T04:48:06.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34682024-12-14T04:48:06.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34692024-12-14T04:48:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34702024-12-14T04:48:06.889ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34712024-12-14T04:48:06.890ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1460, tv_nsec: 987910073 } })
34722024-12-14T04:48:06.890ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
34732024-12-14T04:48:06.890ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34742024-12-14T04:48:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34752024-12-14T04:48:06.891ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34762024-12-14T04:48:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34772024-12-14T04:48:06.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
34782024-12-14T04:48:06.893ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34792024-12-14T04:48:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34802024-12-14T04:48:06.895ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 04:38:06.881648730 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
34812024-12-14T04:48:06.895ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.131048ms iteration = 14 status = {"expiration":"2024-12-14T04:38:06.881648730Z","pruned":[]}
34822024-12-14T04:48:06.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34832024-12-14T04:48:06.897ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 15.381581ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
34842024-12-14T04:48:06.897ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1460, tv_nsec: 995105031 } })
34852024-12-14T04:48:06.898ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
34862024-12-14T04:48:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34872024-12-14T04:48:06.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34882024-12-14T04:48:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34892024-12-14T04:48:06.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34902024-12-14T04:48:06.904ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34912024-12-14T04:48:06.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34922024-12-14T04:48:06.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34932024-12-14T04:48:06.905ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34942024-12-14T04:48:06.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34952024-12-14T04:48:06.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34962024-12-14T04:48:06.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34972024-12-14T04:48:06.910ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34982024-12-14T04:48:06.910ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34992024-12-14T04:48:06.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35002024-12-14T04:48:06.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35012024-12-14T04:48:06.911ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1461, tv_nsec: 8454179 } })
35022024-12-14T04:48:06.911ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
35032024-12-14T04:48:06.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35042024-12-14T04:48:06.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35052024-12-14T04:48:06.912ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35062024-12-14T04:48:06.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
35072024-12-14T04:48:06.915ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35082024-12-14T04:48:06.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35092024-12-14T04:48:06.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35102024-12-14T04:48:06.919ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 38.284808ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
35112024-12-14T04:48:06.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
35122024-12-14T04:48:06.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35132024-12-14T04:48:06.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35142024-12-14T04:48:06.930ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35152024-12-14T04:48:06.930ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35162024-12-14T04:48:06.930ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
35172024-12-14T04:48:06.930ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35182024-12-14T04:48:06.930ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35192024-12-14T04:48:06.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35202024-12-14T04:48:06.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35212024-12-14T04:48:06.931ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35222024-12-14T04:48:06.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35232024-12-14T04:48:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
35242024-12-14T04:48:06.932ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35252024-12-14T04:48:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35262024-12-14T04:48:06.939ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
35272024-12-14T04:48:06.939ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.80087ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
35282024-12-14T04:48:06.946ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35292024-12-14T04:48:06.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35302024-12-14T04:48:06.951ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35312024-12-14T04:48:06.953ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35322024-12-14T04:48:06.955ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35332024-12-14T04:48:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35342024-12-14T04:48:06.955ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35352024-12-14T04:48:06.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35362024-12-14T04:48:06.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35372024-12-14T04:48:06.956ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35382024-12-14T04:48:06.956ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.107737ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35392024-12-14T04:48:06.957ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35402024-12-14T04:48:06.957ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
35412024-12-14T04:48:06.958ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35422024-12-14T04:48:06.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35432024-12-14T04:48:06.958ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35442024-12-14T04:48:06.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35452024-12-14T04:48:06.960ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
35462024-12-14T04:48:06.960ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
35472024-12-14T04:48:06.960ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 78.555497ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
35482024-12-14T04:48:06.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35492024-12-14T04:48:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35502024-12-14T04:48:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
35512024-12-14T04:48:06.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35522024-12-14T04:48:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35532024-12-14T04:48:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35542024-12-14T04:48:06.963ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35552024-12-14T04:48:06.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35562024-12-14T04:48:06.965ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35572024-12-14T04:48:06.965ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 83.639049ms iteration = 27 status = 0
35582024-12-14T04:48:06.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35592024-12-14T04:48:06.970ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35602024-12-14T04:48:06.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35612024-12-14T04:48:06.972ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35622024-12-14T04:48:06.975ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
35632024-12-14T04:48:06.975ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35642024-12-14T04:48:06.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35652024-12-14T04:48:06.984ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35662024-12-14T04:48:06.984ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
35672024-12-14T04:48:06.984ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35682024-12-14T04:48:06.984ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.741317ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
35692024-12-14T04:48:06.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35702024-12-14T04:48:06.989ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
35712024-12-14T04:48:06.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
35722024-12-14T04:48:06.990ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
35732024-12-14T04:48:06.990ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
35742024-12-14T04:48:06.990ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
35752024-12-14T04:48:06.990ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35762024-12-14T04:48:06.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35772024-12-14T04:48:06.991ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
35782024-12-14T04:48:06.991ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 108.802464ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
35792024-12-14T04:48:06.991ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35802024-12-14T04:48:06.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35812024-12-14T04:48:06.994ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
35822024-12-14T04:48:06.994ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
35832024-12-14T04:48:06.994ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
35842024-12-14T04:48:06.994ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35852024-12-14T04:48:06.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35862024-12-14T04:48:06.996ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35872024-12-14T04:48:06.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35882024-12-14T04:48:06.998ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35892024-12-14T04:48:06.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35902024-12-14T04:48:07.003ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 121.699601ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35912024-12-14T04:48:07.014ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
35922024-12-14T04:48:07.014ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35932024-12-14T04:48:07.015ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe40b47c-1f8e-4a9f-aa58-c8a49feeb63f", "content-length": "26", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
35942024-12-14T04:48:07.015ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35952024-12-14T04:48:07.016ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35962024-12-14T04:48:07.019ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
35972024-12-14T04:48:07.019ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T04:35:03.711237Z, time_modified: 2024-12-14T04:35:03.711237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
35982024-12-14T04:48:07.019ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 138.382146ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]}},"default_endpoint":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
35992024-12-14T04:48:07.020ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36002024-12-14T04:48:07.020ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
36012024-12-14T04:48:07.021ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 139.461474ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
36022024-12-14T04:48:07.022ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36032024-12-14T04:48:07.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
36042024-12-14T04:48:07.025ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36052024-12-14T04:48:07.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36062024-12-14T04:48:07.029ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
36072024-12-14T04:48:07.030ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
36082024-12-14T04:48:07.030ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36092024-12-14T04:48:07.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36102024-12-14T04:48:07.031ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36112024-12-14T04:48:07.034ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
36122024-12-14T04:48:07.034ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 152.502994ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
36132024-12-14T04:48:07.034ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36142024-12-14T04:48:07.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36152024-12-14T04:48:07.037ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
36162024-12-14T04:48:07.037ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
36172024-12-14T04:48:07.037ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 04:35:18.025163 UTC dns_group = internal generation = 1
36182024-12-14T04:48:07.037ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 155.643656ms iteration = 15 status = {"generation":1}
36192024-12-14T04:48:07.056ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
36202024-12-14T04:48:07.056ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 0 version = 2
36212024-12-14T04:48:07.056ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36222024-12-14T04:48:07.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36232024-12-14T04:48:07.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36242024-12-14T04:48:07.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
36252024-12-14T04:48:07.060ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36262024-12-14T04:48:07.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36272024-12-14T04:48:07.062ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 1 version = 2
36282024-12-14T04:48:07.062ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
36292024-12-14T04:48:07.062ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 04:35:19.721992 UTC dns_group = external generation = 2
36302024-12-14T04:48:07.062ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 181.350655ms iteration = 15 status = {"generation":2}
36312024-12-14T04:48:07.066ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 185.223686ms iteration = 31 status = {"physical_disks_added":0}
36322024-12-14T04:48:07.068ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36332024-12-14T04:48:07.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36342024-12-14T04:48:07.069ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
36352024-12-14T04:48:07.069ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 187.116455ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
36362024-12-14T04:48:07.070ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36372024-12-14T04:48:07.071ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36382024-12-14T04:48:07.071ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36392024-12-14T04:48:07.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36402024-12-14T04:48:07.073ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36412024-12-14T04:48:07.074ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36422024-12-14T04:48:07.074ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36432024-12-14T04:48:07.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36442024-12-14T04:48:07.099ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
36452024-12-14T04:48:07.099ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36462024-12-14T04:48:07.099ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c8c4916-cc52-4af2-8a3e-80a4a77b5bec", "content-length": "26", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
36472024-12-14T04:48:07.099ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36482024-12-14T04:48:07.101ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
36492024-12-14T04:48:07.131ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
36502024-12-14T04:48:07.203ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
36512024-12-14T04:48:07.203ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36522024-12-14T04:48:07.204ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0742d205-9b04-4991-a42b-e576317f7f30", "content-length": "26", "date": "Sat, 14 Dec 2024 04:48:07 GMT"} })
36532024-12-14T04:48:07.204ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36542024-12-14T04:48:07.205ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36552024-12-14T04:48:07.212ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
36562024-12-14T04:48:07.212ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "27fba5fa-a410-4878-8317-79325fde1393", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
36572024-12-14T04:48:07.212ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
36582024-12-14T04:48:07.214ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b622d2c9-9fa3-4429-9319-4e8844a8371b", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
36592024-12-14T04:48:07.214ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
36602024-12-14T04:48:07.215ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
36612024-12-14T04:48:07.215ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 333.737436ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
36622024-12-14T04:48:07.320ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
36632024-12-14T04:48:07.320ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36642024-12-14T04:48:07.320ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d962a0fa-d8b8-44ac-bfdd-a03ab393ea14", "content-length": "372", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
36652024-12-14T04:48:07.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36662024-12-14T04:48:07.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36672024-12-14T04:48:07.323ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 441.418652ms iteration = 27 status = {}
36682024-12-14T04:48:07.337ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
36692024-12-14T04:48:07.337ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
36702024-12-14T04:48:07.337ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6000e68c-12e7-4a6b-a0a1-a6b5159dd82e", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
36712024-12-14T04:48:07.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36722024-12-14T04:48:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36732024-12-14T04:48:07.342ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36742024-12-14T04:48:07.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36752024-12-14T04:48:07.345ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36762024-12-14T04:48:07.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36772024-12-14T04:48:07.349ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
36782024-12-14T04:48:07.403ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
36792024-12-14T04:48:07.403ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c23b75-d21c-45b6-be63-343522f62d09", "content-length": "2", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
36802024-12-14T04:48:07.405ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36812024-12-14T04:48:07.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36822024-12-14T04:48:07.424ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
36832024-12-14T04:48:07.425ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "14ab8eaa-fadf-411e-9d25-61c1cf3896f4", "content-length": "691", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
36842024-12-14T04:48:07.425ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
36852024-12-14T04:48:07.425ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
36862024-12-14T04:48:07.425ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
36872024-12-14T04:48:07.426ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 545.175165ms iteration = 28 status = {}
36882024-12-14T04:48:07.427ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 545.809492ms iteration = 27 status = null
36892024-12-14T04:48:07.428ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
36902024-12-14T04:48:07.428ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36912024-12-14T04:48:07.428ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7d59f610-dcfe-4fb2-917e-8a8c65ccd36f", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
36922024-12-14T04:48:07.430ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 548.485852ms iteration = 29 status = {}
36932024-12-14T04:48:07.432ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3993aa5e-61ca-4c54-9412-e66cc081952b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
36942024-12-14T04:48:07.432ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3993aa5e-61ca-4c54-9412-e66cc081952b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
36952024-12-14T04:48:07.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1926 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3993aa5e-61ca-4c54-9412-e66cc081952b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
36962024-12-14T04:48:07.446ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
36972024-12-14T04:48:07.447ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e5d2696c-64b4-4e58-9eb6-9575e50f2e7a", "date": "Sat, 14 Dec 2024 04:48:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
36982024-12-14T04:48:07.447ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
36992024-12-14T04:48:07.449ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a56c7cfc-b1a2-43ae-977e-ebbc4aa70b29", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
37002024-12-14T04:48:07.449ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
37012024-12-14T04:48:07.451ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "01875d19-c9d9-47ba-a7b9-33f3813ef250", "date": "Sat, 14 Dec 2024 04:48:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
37022024-12-14T04:48:07.451ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
37032024-12-14T04:48:07.453ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
37042024-12-14T04:48:07.453ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 572.22014ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
37052024-12-14T04:48:07.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37062024-12-14T04:48:07.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37072024-12-14T04:48:07.500ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
37082024-12-14T04:48:07.500ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37092024-12-14T04:48:07.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37102024-12-14T04:48:07.510ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
37112024-12-14T04:48:07.510ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "eab2b7ce-7391-4553-84a7-2dcce2967442", "content-length": "134", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
37122024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
37132024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
37142024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
37152024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
37162024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37172024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37182024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37192024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37202024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37212024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37222024-12-14T04:48:07.511ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
37232024-12-14T04:48:07.511ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37242024-12-14T04:48:07.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37252024-12-14T04:48:07.514ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
37262024-12-14T04:48:07.514ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "11f846ea-d96e-4faf-be30-e6ae8274d35e", "content-length": "2", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
37272024-12-14T04:48:07.514ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
37282024-12-14T04:48:07.514ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c447f76-d0e2-4f30-bb99-6c2a446f4f80", "content-length": "52", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
37292024-12-14T04:48:07.514ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
37302024-12-14T04:48:07.514ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
37312024-12-14T04:48:07.608ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a8e7dcd5-b64c-4c53-a821-4113ee177779", "date": "Sat, 14 Dec 2024 04:48:07 GMT"} })
37322024-12-14T04:48:07.608ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
37332024-12-14T04:48:07.608ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
37342024-12-14T04:48:07.608ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6234f880-d017-4e69-8fba-4e0020222e92", "content-length": "682", "date": "Sat, 14 Dec 2024 04:48:06 GMT"} })
37352024-12-14T04:48:07.608ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37362024-12-14T04:48:07.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37372024-12-14T04:48:07.612ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37382024-12-14T04:48:07.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37392024-12-14T04:48:07.616ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37402024-12-14T04:48:07.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37412024-12-14T04:48:07.618ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37422024-12-14T04:48:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37432024-12-14T04:48:07.625ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 743.586005ms iteration = 27 status = {}
37442024-12-14T04:48:07.752ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:60896
37452024-12-14T04:48:07.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60896 req_id = 5327c4eb-0f9a-48ed-a9ec-ccf05974e99c roles = RoleSet { roles: {} } uri = /metrics/producers
37462024-12-14T04:48:07.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60896 req_id = 5327c4eb-0f9a-48ed-a9ec-ccf05974e99c resource = Database result = Ok(()) uri = /metrics/producers
37472024-12-14T04:48:07.793ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 7c8e60d1-cbc0-4976-b68e-162aaf207501
37482024-12-14T04:48:07.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40729 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60896 req_id = 5327c4eb-0f9a-48ed-a9ec-ccf05974e99c response_code = 201 uri = /metrics/producers
37492024-12-14T04:48:08.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11906e60-3673-4f89-b409-28657d590d0a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37502024-12-14T04:48:08.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11906e60-3673-4f89-b409-28657d590d0a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37512024-12-14T04:48:08.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1977 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11906e60-3673-4f89-b409-28657d590d0a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37522024-12-14T04:48:09.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3f2b046-c9fc-486a-b9ce-ce273257ae55 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37532024-12-14T04:48:09.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3f2b046-c9fc-486a-b9ce-ce273257ae55 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37542024-12-14T04:48:09.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2214 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3f2b046-c9fc-486a-b9ce-ce273257ae55 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37552024-12-14T04:48:10.498ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 58e8b241-a140-43ab-b39b-2ba93191fcde roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37562024-12-14T04:48:10.499ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 58e8b241-a140-43ab-b39b-2ba93191fcde resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37572024-12-14T04:48:10.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1972 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 58e8b241-a140-43ab-b39b-2ba93191fcde response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37582024-12-14T04:48:11.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4471e0e1-0bd8-49fe-aea9-8293a3558edd roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37592024-12-14T04:48:11.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4471e0e1-0bd8-49fe-aea9-8293a3558edd resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37602024-12-14T04:48:11.521ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2486 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4471e0e1-0bd8-49fe-aea9-8293a3558edd response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37612024-12-14T04:48:12.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8cb7fca7-805f-484f-9342-609f1fa04771 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37622024-12-14T04:48:12.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8cb7fca7-805f-484f-9342-609f1fa04771 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37632024-12-14T04:48:12.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2622 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8cb7fca7-805f-484f-9342-609f1fa04771 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37642024-12-14T04:48:12.783ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1487 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 0a7089ea-847e-47f3-bf02-b5c154d28f74 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
37652024-12-14T04:48:13.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 44c247bd-5aac-402e-9f5b-f110b2f0e41e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37662024-12-14T04:48:13.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 44c247bd-5aac-402e-9f5b-f110b2f0e41e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37672024-12-14T04:48:13.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2303 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 44c247bd-5aac-402e-9f5b-f110b2f0e41e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37682024-12-14T04:48:14.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 45259550-3474-467e-9df0-62d4b2a8bfb3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37692024-12-14T04:48:14.585ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 45259550-3474-467e-9df0-62d4b2a8bfb3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37702024-12-14T04:48:14.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2526 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 45259550-3474-467e-9df0-62d4b2a8bfb3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37712024-12-14T04:48:15.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9b9d7dd9-10e6-4e7b-8488-3751bf3ce2cc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37722024-12-14T04:48:15.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9b9d7dd9-10e6-4e7b-8488-3751bf3ce2cc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37732024-12-14T04:48:15.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9b9d7dd9-10e6-4e7b-8488-3751bf3ce2cc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37742024-12-14T04:48:16.629ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c644a0c6-9dc8-4e3e-8b02-637ff4bd29bc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37752024-12-14T04:48:16.629ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c644a0c6-9dc8-4e3e-8b02-637ff4bd29bc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37762024-12-14T04:48:16.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2394 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c644a0c6-9dc8-4e3e-8b02-637ff4bd29bc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37772024-12-14T04:48:16.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
37782024-12-14T04:48:16.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37792024-12-14T04:48:16.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37802024-12-14T04:48:16.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
37812024-12-14T04:48:16.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37822024-12-14T04:48:16.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37832024-12-14T04:48:16.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37842024-12-14T04:48:16.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37852024-12-14T04:48:16.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37862024-12-14T04:48:16.886ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.378774ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37872024-12-14T04:48:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37882024-12-14T04:48:16.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37892024-12-14T04:48:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37902024-12-14T04:48:16.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37912024-12-14T04:48:16.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37922024-12-14T04:48:16.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37932024-12-14T04:48:16.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
37942024-12-14T04:48:16.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37952024-12-14T04:48:16.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37962024-12-14T04:48:16.905ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
37972024-12-14T04:48:16.905ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.079075ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
37982024-12-14T04:48:17.652ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a1a94573-416b-442b-a537-1e74241a5050 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
37992024-12-14T04:48:17.652ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a1a94573-416b-442b-a537-1e74241a5050 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38002024-12-14T04:48:17.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2504 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a1a94573-416b-442b-a537-1e74241a5050 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38012024-12-14T04:48:18.674ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a5032c2d-6371-4557-a233-ac57d161542e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38022024-12-14T04:48:18.675ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a5032c2d-6371-4557-a233-ac57d161542e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38032024-12-14T04:48:18.677ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2501 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a5032c2d-6371-4557-a233-ac57d161542e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38042024-12-14T04:48:19.697ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2b9dfc2d-8fc4-4b61-a213-42bd72d528b1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38052024-12-14T04:48:19.697ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2b9dfc2d-8fc4-4b61-a213-42bd72d528b1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38062024-12-14T04:48:19.699ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1978 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2b9dfc2d-8fc4-4b61-a213-42bd72d528b1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38072024-12-14T04:48:20.718ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d76c74b0-b077-44d1-a0ed-8a6ead25f695 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38082024-12-14T04:48:20.719ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d76c74b0-b077-44d1-a0ed-8a6ead25f695 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38092024-12-14T04:48:20.721ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2403 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d76c74b0-b077-44d1-a0ed-8a6ead25f695 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38102024-12-14T04:48:21.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 350e2e7f-3a3d-4417-99ed-4821b21a5566 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38112024-12-14T04:48:21.742ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 350e2e7f-3a3d-4417-99ed-4821b21a5566 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38122024-12-14T04:48:21.744ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2342 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 350e2e7f-3a3d-4417-99ed-4821b21a5566 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38132024-12-14T04:48:22.763ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1dec30d5-85a2-4a7c-9e01-9d27cdba9cc9 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38142024-12-14T04:48:22.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1dec30d5-85a2-4a7c-9e01-9d27cdba9cc9 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38152024-12-14T04:48:22.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2124 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1dec30d5-85a2-4a7c-9e01-9d27cdba9cc9 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38162024-12-14T04:48:22.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1333 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 6c6c49a5-8c5a-4b82-bc14-16d803dfac14 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
38172024-12-14T04:48:23.785ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca284891-4840-47e5-8671-9257d2c8a086 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38182024-12-14T04:48:23.785ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca284891-4840-47e5-8671-9257d2c8a086 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38192024-12-14T04:48:23.787ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2364 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ca284891-4840-47e5-8671-9257d2c8a086 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38202024-12-14T04:48:24.124ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:38386
38212024-12-14T04:48:24.124ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38386 req_id = 6776ffce-5932-441d-a983-ff528fbeea33 roles = RoleSet { roles: {} } uri = /metrics/producers
38222024-12-14T04:48:24.125ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38386 req_id = 6776ffce-5932-441d-a983-ff528fbeea33 resource = Database result = Ok(()) uri = /metrics/producers
38232024-12-14T04:48:24.134ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 4f6cecdf-822e-4790-965b-a20901bfe59f
38242024-12-14T04:48:24.134ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9630 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38386 req_id = 6776ffce-5932-441d-a983-ff528fbeea33 response_code = 201 uri = /metrics/producers
38252024-12-14T04:48:24.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a87ce239-44ab-45b8-9b79-9e06e0d9a51b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38262024-12-14T04:48:24.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a87ce239-44ab-45b8-9b79-9e06e0d9a51b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38272024-12-14T04:48:24.807ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2052 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a87ce239-44ab-45b8-9b79-9e06e0d9a51b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38282024-12-14T04:48:25.827ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6611293b-0a7e-43c5-adbc-4e606ce48270 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38292024-12-14T04:48:25.827ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6611293b-0a7e-43c5-adbc-4e606ce48270 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38302024-12-14T04:48:25.829ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2395 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6611293b-0a7e-43c5-adbc-4e606ce48270 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38312024-12-14T04:48:26.582ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:54712
38322024-12-14T04:48:26.583ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54712 req_id = 086e9ad5-2a0e-4c8c-bc71-92ddc2ca98c7 roles = RoleSet { roles: {} } uri = /metrics/producers
38332024-12-14T04:48:26.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54712 req_id = 086e9ad5-2a0e-4c8c-bc71-92ddc2ca98c7 resource = Database result = Ok(()) uri = /metrics/producers
38342024-12-14T04:48:26.594ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38352024-12-14T04:48:26.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11124 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54712 req_id = 086e9ad5-2a0e-4c8c-bc71-92ddc2ca98c7 response_code = 201 uri = /metrics/producers
38362024-12-14T04:48:26.849ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 85f68795-914d-41e2-bf49-5bbeb1d0f3a3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38372024-12-14T04:48:26.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 85f68795-914d-41e2-bf49-5bbeb1d0f3a3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38382024-12-14T04:48:26.851ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2352 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 85f68795-914d-41e2-bf49-5bbeb1d0f3a3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38392024-12-14T04:48:26.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
38402024-12-14T04:48:26.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38412024-12-14T04:48:26.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38422024-12-14T04:48:26.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
38432024-12-14T04:48:26.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38442024-12-14T04:48:26.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38452024-12-14T04:48:26.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38462024-12-14T04:48:26.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38472024-12-14T04:48:26.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38482024-12-14T04:48:26.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.637491ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38492024-12-14T04:48:26.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38502024-12-14T04:48:26.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38512024-12-14T04:48:26.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38522024-12-14T04:48:26.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38532024-12-14T04:48:26.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38542024-12-14T04:48:26.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38552024-12-14T04:48:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
38562024-12-14T04:48:26.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38572024-12-14T04:48:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38582024-12-14T04:48:26.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
38592024-12-14T04:48:26.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.278863ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
38602024-12-14T04:48:27.873ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2a31d58d-c6bd-488b-a926-3ae21aeeb916 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38612024-12-14T04:48:27.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2a31d58d-c6bd-488b-a926-3ae21aeeb916 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38622024-12-14T04:48:27.875ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2301 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2a31d58d-c6bd-488b-a926-3ae21aeeb916 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38632024-12-14T04:48:28.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1422e836-ba77-483c-a61e-b7117989c532 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38642024-12-14T04:48:28.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1422e836-ba77-483c-a61e-b7117989c532 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38652024-12-14T04:48:28.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2304 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1422e836-ba77-483c-a61e-b7117989c532 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38662024-12-14T04:48:29.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5128d435-c378-4ac8-873b-ff50d515e63c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38672024-12-14T04:48:29.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5128d435-c378-4ac8-873b-ff50d515e63c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38682024-12-14T04:48:29.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2543 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5128d435-c378-4ac8-873b-ff50d515e63c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38692024-12-14T04:48:30.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 03a66820-3399-438a-a3d3-a107ce29aaee roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38702024-12-14T04:48:30.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 03a66820-3399-438a-a3d3-a107ce29aaee resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38712024-12-14T04:48:30.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2532 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 03a66820-3399-438a-a3d3-a107ce29aaee response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38722024-12-14T04:48:31.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2b198385-caad-4f5b-b17d-6898f6eabd55 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38732024-12-14T04:48:31.962ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2b198385-caad-4f5b-b17d-6898f6eabd55 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38742024-12-14T04:48:31.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1961 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2b198385-caad-4f5b-b17d-6898f6eabd55 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38752024-12-14T04:48:32.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1473 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = cb528f5e-8298-45da-8bfd-19960122dcee response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
38762024-12-14T04:48:32.983ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7729aaf6-0253-4151-b71e-ab33f5562c0f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38772024-12-14T04:48:32.983ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7729aaf6-0253-4151-b71e-ab33f5562c0f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38782024-12-14T04:48:32.985ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2625 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7729aaf6-0253-4151-b71e-ab33f5562c0f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38792024-12-14T04:48:34.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2160ddc5-e5cc-47aa-9184-588aeb1541ee roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38802024-12-14T04:48:34.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2160ddc5-e5cc-47aa-9184-588aeb1541ee resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38812024-12-14T04:48:34.008ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2191 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2160ddc5-e5cc-47aa-9184-588aeb1541ee response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38822024-12-14T04:48:35.028ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d3dff0a1-c981-4f5b-ac69-0bf8f1f6b32e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38832024-12-14T04:48:35.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d3dff0a1-c981-4f5b-ac69-0bf8f1f6b32e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38842024-12-14T04:48:35.030ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2768 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d3dff0a1-c981-4f5b-ac69-0bf8f1f6b32e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38852024-12-14T04:48:36.050ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a3b19a8b-a89e-433c-9d08-846b4b2058f8 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38862024-12-14T04:48:36.050ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a3b19a8b-a89e-433c-9d08-846b4b2058f8 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38872024-12-14T04:48:36.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2184 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a3b19a8b-a89e-433c-9d08-846b4b2058f8 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
38882024-12-14T04:48:36.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = f912f8b4-fa7a-43e1-8e9e-c39fa398021e roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
38892024-12-14T04:48:36.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = f912f8b4-fa7a-43e1-8e9e-c39fa398021e resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
38902024-12-14T04:48:36.861ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3983 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = f912f8b4-fa7a-43e1-8e9e-c39fa398021e response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
38912024-12-14T04:48:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
38922024-12-14T04:48:36.882ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
38932024-12-14T04:48:36.883ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
38942024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
38952024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
38962024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38972024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
38982024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
38992024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
39002024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39012024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
39022024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
39032024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39042024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
39052024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
39062024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39072024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
39082024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39092024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
39102024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39112024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
39122024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39132024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
39142024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
39152024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39162024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
39172024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39182024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
39192024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39202024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
39212024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39222024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
39232024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
39242024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39252024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39262024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39272024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39282024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
39292024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
39302024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39312024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39322024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39332024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39342024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39352024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
39362024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39372024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39382024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39392024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39402024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39412024-12-14T04:48:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
39422024-12-14T04:48:36.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39432024-12-14T04:48:36.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39442024-12-14T04:48:36.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39452024-12-14T04:48:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39462024-12-14T04:48:36.887ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.082216ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
39472024-12-14T04:48:36.887ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1490, tv_nsec: 983679323 } })
39482024-12-14T04:48:36.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39492024-12-14T04:48:36.887ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
39502024-12-14T04:48:36.888ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39512024-12-14T04:48:36.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
39522024-12-14T04:48:36.889ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39532024-12-14T04:48:36.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39542024-12-14T04:48:36.890ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1490, tv_nsec: 986644340 } })
39552024-12-14T04:48:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39562024-12-14T04:48:36.890ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39572024-12-14T04:48:36.890ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
39582024-12-14T04:48:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39592024-12-14T04:48:36.892ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39602024-12-14T04:48:36.893ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39612024-12-14T04:48:36.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39622024-12-14T04:48:36.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39632024-12-14T04:48:36.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39642024-12-14T04:48:36.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39652024-12-14T04:48:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39662024-12-14T04:48:36.895ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39672024-12-14T04:48:36.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39682024-12-14T04:48:36.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39692024-12-14T04:48:36.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39702024-12-14T04:48:36.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39712024-12-14T04:48:36.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39722024-12-14T04:48:36.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39732024-12-14T04:48:36.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39742024-12-14T04:48:36.900ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1490, tv_nsec: 996594890 } })
39752024-12-14T04:48:36.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39762024-12-14T04:48:36.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39772024-12-14T04:48:36.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39782024-12-14T04:48:36.900ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
39792024-12-14T04:48:36.901ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39802024-12-14T04:48:36.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39812024-12-14T04:48:36.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39822024-12-14T04:48:36.903ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39832024-12-14T04:48:36.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39842024-12-14T04:48:36.904ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
39852024-12-14T04:48:36.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39862024-12-14T04:48:36.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39872024-12-14T04:48:36.919ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
39882024-12-14T04:48:36.920ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 36.82684ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
39892024-12-14T04:48:36.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 38.174294ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
39902024-12-14T04:48:36.930ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 47.5631ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
39912024-12-14T04:48:36.932ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39922024-12-14T04:48:36.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39932024-12-14T04:48:36.933ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39942024-12-14T04:48:36.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39952024-12-14T04:48:36.935ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39962024-12-14T04:48:36.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39972024-12-14T04:48:36.935ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39982024-12-14T04:48:36.936ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39992024-12-14T04:48:36.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40002024-12-14T04:48:36.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40012024-12-14T04:48:36.936ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40022024-12-14T04:48:36.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40032024-12-14T04:48:36.937ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40042024-12-14T04:48:36.938ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40052024-12-14T04:48:36.938ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
40062024-12-14T04:48:36.938ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40072024-12-14T04:48:36.938ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 55.895667ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40082024-12-14T04:48:36.939ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40092024-12-14T04:48:36.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40102024-12-14T04:48:36.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40112024-12-14T04:48:36.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
40122024-12-14T04:48:36.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40132024-12-14T04:48:36.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40142024-12-14T04:48:36.941ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40152024-12-14T04:48:36.941ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
40162024-12-14T04:48:36.941ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40172024-12-14T04:48:36.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40182024-12-14T04:48:36.943ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 60.024364ms iteration = 28 status = 0
40192024-12-14T04:48:36.944ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40202024-12-14T04:48:36.944ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
40212024-12-14T04:48:36.944ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 61.615245ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
40222024-12-14T04:48:36.949ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40232024-12-14T04:48:36.949ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.78747ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40242024-12-14T04:48:36.950ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
40252024-12-14T04:48:36.950ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40262024-12-14T04:48:36.951ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b19b2121-44fe-4840-88bd-4798aed7852a", "content-length": "26", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40272024-12-14T04:48:36.951ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40282024-12-14T04:48:36.951ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40292024-12-14T04:48:36.952ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40302024-12-14T04:48:36.954ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40312024-12-14T04:48:36.954ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40322024-12-14T04:48:36.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40332024-12-14T04:48:36.959ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
40342024-12-14T04:48:36.959ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40352024-12-14T04:48:36.960ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "684a4875-b7da-468b-b273-168b0d4d56ab", "content-length": "26", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40362024-12-14T04:48:36.960ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40372024-12-14T04:48:36.962ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40382024-12-14T04:48:36.967ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 84.166784ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
40392024-12-14T04:48:36.967ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.674436ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40402024-12-14T04:48:36.970ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 87.011437ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
40412024-12-14T04:48:36.979ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40422024-12-14T04:48:36.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40432024-12-14T04:48:36.999ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
40442024-12-14T04:48:36.999ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40452024-12-14T04:48:36.999ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a03ebf7-f9ff-4c32-9035-5846cbaef2a5", "content-length": "26", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40462024-12-14T04:48:36.999ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40472024-12-14T04:48:37.001ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
40482024-12-14T04:48:37.021ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 137.945914ms iteration = 32 status = {"physical_disks_added":0}
40492024-12-14T04:48:37.034ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 151.436756ms iteration = 28 status = {}
40502024-12-14T04:48:37.035ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
40512024-12-14T04:48:37.036ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7463ddc-792f-476f-8a3b-5ef0ae142ac2", "content-length": "2", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40522024-12-14T04:48:37.038ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40532024-12-14T04:48:37.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40542024-12-14T04:48:37.039ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
40552024-12-14T04:48:37.039ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40562024-12-14T04:48:37.040ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b561a9e-db95-46b3-9d49-76cf9c285d9f", "content-length": "372", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40572024-12-14T04:48:37.040ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 157.130214ms iteration = 28 status = null
40582024-12-14T04:48:37.040ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40592024-12-14T04:48:37.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40602024-12-14T04:48:37.055ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
40612024-12-14T04:48:37.055ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
40622024-12-14T04:48:37.055ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4c1a2f88-84d5-4f07-9649-429421a252ed", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40632024-12-14T04:48:37.055ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40642024-12-14T04:48:37.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40652024-12-14T04:48:37.057ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40662024-12-14T04:48:37.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40672024-12-14T04:48:37.061ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40682024-12-14T04:48:37.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40692024-12-14T04:48:37.064ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
40702024-12-14T04:48:37.064ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40712024-12-14T04:48:37.065ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "193a7291-29c7-42e5-abcb-e0b518d5d40b", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40722024-12-14T04:48:37.066ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 183.301666ms iteration = 30 status = {}
40732024-12-14T04:48:37.076ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40742024-12-14T04:48:37.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40752024-12-14T04:48:37.077ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40762024-12-14T04:48:37.079ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
40772024-12-14T04:48:37.079ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40782024-12-14T04:48:37.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40792024-12-14T04:48:37.079ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1c7f74b2-0c34-40f7-893b-555ef675e35b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
40802024-12-14T04:48:37.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1c7f74b2-0c34-40f7-893b-555ef675e35b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
40812024-12-14T04:48:37.080ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40822024-12-14T04:48:37.081ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1755 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1c7f74b2-0c34-40f7-893b-555ef675e35b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
40832024-12-14T04:48:37.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
40842024-12-14T04:48:37.082ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40852024-12-14T04:48:37.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40862024-12-14T04:48:37.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40872024-12-14T04:48:37.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40882024-12-14T04:48:37.143ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
40892024-12-14T04:48:37.143ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
40902024-12-14T04:48:37.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40912024-12-14T04:48:37.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40922024-12-14T04:48:37.144ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e62767be-b58d-4520-b0c3-004b53eb7962", "content-length": "691", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40932024-12-14T04:48:37.144ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
40942024-12-14T04:48:37.144ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
40952024-12-14T04:48:37.144ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
40962024-12-14T04:48:37.145ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 262.918021ms iteration = 29 status = {}
40972024-12-14T04:48:37.151ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
40982024-12-14T04:48:37.151ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "637bcb00-c8e7-4966-84b8-84131c08e601", "content-length": "134", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
40992024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
41002024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
41012024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
41022024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
41032024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41042024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41052024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41062024-12-14T04:48:37.151ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41072024-12-14T04:48:37.152ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41082024-12-14T04:48:37.152ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41092024-12-14T04:48:37.152ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
41102024-12-14T04:48:37.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41112024-12-14T04:48:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41122024-12-14T04:48:37.154ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
41132024-12-14T04:48:37.154ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec45f811-5370-419d-8bfa-f36d50883233", "content-length": "2", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
41142024-12-14T04:48:37.154ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
41152024-12-14T04:48:37.154ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9b61e18-b0f8-484f-83ba-2a7c1df04dc0", "content-length": "52", "date": "Sat, 14 Dec 2024 04:48:36 GMT"} })
41162024-12-14T04:48:37.154ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
41172024-12-14T04:48:37.154ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41182024-12-14T04:48:37.248ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8c526ac6-cbef-4631-b53e-1da7a29aa87b", "date": "Sat, 14 Dec 2024 04:48:37 GMT"} })
41192024-12-14T04:48:37.248ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
41202024-12-14T04:48:37.248ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
41212024-12-14T04:48:37.249ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "082d5c11-36ac-43a0-8931-fe994784d74b", "content-length": "682", "date": "Sat, 14 Dec 2024 04:48:37 GMT"} })
41222024-12-14T04:48:37.249ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41232024-12-14T04:48:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41242024-12-14T04:48:37.252ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41252024-12-14T04:48:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41262024-12-14T04:48:37.255ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41272024-12-14T04:48:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41282024-12-14T04:48:37.257ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41292024-12-14T04:48:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41302024-12-14T04:48:37.264ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 381.381428ms iteration = 28 status = {}
41312024-12-14T04:48:38.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1896d73d-ee33-4855-b7de-acff391e3f17 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41322024-12-14T04:48:38.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1896d73d-ee33-4855-b7de-acff391e3f17 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41332024-12-14T04:48:38.103ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2151 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1896d73d-ee33-4855-b7de-acff391e3f17 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41342024-12-14T04:48:39.123ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83133e35-9a41-4f96-99c3-d3e41dea0743 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41352024-12-14T04:48:39.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83133e35-9a41-4f96-99c3-d3e41dea0743 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41362024-12-14T04:48:39.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2175 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 83133e35-9a41-4f96-99c3-d3e41dea0743 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41372024-12-14T04:48:40.145ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7233a7bd-c64c-438b-a4d8-a153e9774004 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41382024-12-14T04:48:40.145ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7233a7bd-c64c-438b-a4d8-a153e9774004 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41392024-12-14T04:48:40.147ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1975 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7233a7bd-c64c-438b-a4d8-a153e9774004 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41402024-12-14T04:48:41.166ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 783d3413-6e4c-4290-926e-e4b4b5c6e56a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41412024-12-14T04:48:41.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 783d3413-6e4c-4290-926e-e4b4b5c6e56a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41422024-12-14T04:48:41.167ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1894 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 783d3413-6e4c-4290-926e-e4b4b5c6e56a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41432024-12-14T04:48:42.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fc7916a3-db6a-45d4-9286-95110bb6a112 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41442024-12-14T04:48:42.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fc7916a3-db6a-45d4-9286-95110bb6a112 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41452024-12-14T04:48:42.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2499 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fc7916a3-db6a-45d4-9286-95110bb6a112 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41462024-12-14T04:48:42.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1362 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = b57b6f5b-eb60-4d5b-af51-121e168a0049 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
41472024-12-14T04:48:43.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0de36cb1-f9b5-4f6a-8226-5b09f7becae5 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41482024-12-14T04:48:43.212ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0de36cb1-f9b5-4f6a-8226-5b09f7becae5 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41492024-12-14T04:48:43.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2674 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0de36cb1-f9b5-4f6a-8226-5b09f7becae5 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41502024-12-14T04:48:44.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3d57f976-ca58-4668-b516-6ded16a734f0 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41512024-12-14T04:48:44.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3d57f976-ca58-4668-b516-6ded16a734f0 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41522024-12-14T04:48:44.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2364 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3d57f976-ca58-4668-b516-6ded16a734f0 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41532024-12-14T04:48:45.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3b04cba-d8dd-4227-a465-8a60fc06787f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41542024-12-14T04:48:45.256ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3b04cba-d8dd-4227-a465-8a60fc06787f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41552024-12-14T04:48:45.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2243 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b3b04cba-d8dd-4227-a465-8a60fc06787f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41562024-12-14T04:48:46.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2e18d85c-1be9-43bb-a3f2-cc6adbf9e19b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41572024-12-14T04:48:46.279ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2e18d85c-1be9-43bb-a3f2-cc6adbf9e19b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41582024-12-14T04:48:46.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1924 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 2e18d85c-1be9-43bb-a3f2-cc6adbf9e19b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41592024-12-14T04:48:46.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
41602024-12-14T04:48:46.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41612024-12-14T04:48:46.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41622024-12-14T04:48:46.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
41632024-12-14T04:48:46.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41642024-12-14T04:48:46.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41652024-12-14T04:48:46.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41662024-12-14T04:48:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41672024-12-14T04:48:46.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41682024-12-14T04:48:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41692024-12-14T04:48:46.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41702024-12-14T04:48:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41712024-12-14T04:48:46.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41722024-12-14T04:48:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
41732024-12-14T04:48:46.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41742024-12-14T04:48:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41752024-12-14T04:48:46.903ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
41762024-12-14T04:48:46.903ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.31271ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
41772024-12-14T04:48:46.913ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41782024-12-14T04:48:46.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41792024-12-14T04:48:46.914ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.977653ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41802024-12-14T04:48:47.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b093d6f-998c-45da-b9b3-1c97790c79db roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41812024-12-14T04:48:47.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b093d6f-998c-45da-b9b3-1c97790c79db resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41822024-12-14T04:48:47.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2261 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b093d6f-998c-45da-b9b3-1c97790c79db response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41832024-12-14T04:48:48.325ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b687bb18-bd2e-4001-a11e-a57f69530ffb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41842024-12-14T04:48:48.325ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b687bb18-bd2e-4001-a11e-a57f69530ffb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41852024-12-14T04:48:48.327ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2110 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b687bb18-bd2e-4001-a11e-a57f69530ffb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41862024-12-14T04:48:49.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1b20528-a668-4cc0-98de-151ac3a3f7d9 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41872024-12-14T04:48:49.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1b20528-a668-4cc0-98de-151ac3a3f7d9 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41882024-12-14T04:48:49.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2787 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c1b20528-a668-4cc0-98de-151ac3a3f7d9 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41892024-12-14T04:48:50.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4f8b089f-4af1-414f-9fb4-00eaa4929d7f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41902024-12-14T04:48:50.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4f8b089f-4af1-414f-9fb4-00eaa4929d7f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41912024-12-14T04:48:50.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1953 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4f8b089f-4af1-414f-9fb4-00eaa4929d7f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41922024-12-14T04:48:51.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a6b96489-23c3-4c8f-81fb-db4c11b97bdf roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41932024-12-14T04:48:51.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a6b96489-23c3-4c8f-81fb-db4c11b97bdf resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41942024-12-14T04:48:51.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2263 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a6b96489-23c3-4c8f-81fb-db4c11b97bdf response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41952024-12-14T04:48:52.414ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ab804fef-945f-4029-b919-2b4fd2ff4476 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41962024-12-14T04:48:52.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ab804fef-945f-4029-b919-2b4fd2ff4476 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41972024-12-14T04:48:52.416ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1906 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ab804fef-945f-4029-b919-2b4fd2ff4476 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
41982024-12-14T04:48:52.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1356 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 2e8e5924-8790-4c9f-8f69-1eec6fc258f2 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
41992024-12-14T04:48:53.438ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8368276d-5fea-4fcd-8014-9f8c8a90e48f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42002024-12-14T04:48:53.439ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8368276d-5fea-4fcd-8014-9f8c8a90e48f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42012024-12-14T04:48:53.441ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2522 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8368276d-5fea-4fcd-8014-9f8c8a90e48f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42022024-12-14T04:48:54.461ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 98007372-5783-479e-9af6-0712affb5d07 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42032024-12-14T04:48:54.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 98007372-5783-479e-9af6-0712affb5d07 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42042024-12-14T04:48:54.464ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2416 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 98007372-5783-479e-9af6-0712affb5d07 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42052024-12-14T04:48:55.482ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8227e1bb-11ce-4343-b248-bf7e6a49324a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42062024-12-14T04:48:55.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8227e1bb-11ce-4343-b248-bf7e6a49324a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42072024-12-14T04:48:55.484ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2065 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8227e1bb-11ce-4343-b248-bf7e6a49324a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42082024-12-14T04:48:56.504ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f2a2ba43-f4d3-42bf-90b9-eb593f34db56 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42092024-12-14T04:48:56.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f2a2ba43-f4d3-42bf-90b9-eb593f34db56 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42102024-12-14T04:48:56.506ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2170 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f2a2ba43-f4d3-42bf-90b9-eb593f34db56 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42112024-12-14T04:48:56.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
42122024-12-14T04:48:56.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42132024-12-14T04:48:56.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42142024-12-14T04:48:56.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
42152024-12-14T04:48:56.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42162024-12-14T04:48:56.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42172024-12-14T04:48:56.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42182024-12-14T04:48:56.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42192024-12-14T04:48:56.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42202024-12-14T04:48:56.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.279451ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42212024-12-14T04:48:56.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42222024-12-14T04:48:56.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42232024-12-14T04:48:56.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42242024-12-14T04:48:56.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42252024-12-14T04:48:56.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42262024-12-14T04:48:56.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42272024-12-14T04:48:56.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
42282024-12-14T04:48:56.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42292024-12-14T04:48:56.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42302024-12-14T04:48:56.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
42312024-12-14T04:48:56.903ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.59274ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
42322024-12-14T04:48:57.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8f2f6ccb-7408-4d34-ba4f-0cd6912b2fc5 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42332024-12-14T04:48:57.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8f2f6ccb-7408-4d34-ba4f-0cd6912b2fc5 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42342024-12-14T04:48:57.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2153 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8f2f6ccb-7408-4d34-ba4f-0cd6912b2fc5 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42352024-12-14T04:48:58.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b7e92fd-dc0a-4cd1-a7a1-38bd3967c259 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42362024-12-14T04:48:58.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b7e92fd-dc0a-4cd1-a7a1-38bd3967c259 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42372024-12-14T04:48:58.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2163 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b7e92fd-dc0a-4cd1-a7a1-38bd3967c259 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42382024-12-14T04:48:59.571ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 86b5eac5-7649-427a-a1e1-1a1e5e349e73 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42392024-12-14T04:48:59.572ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 86b5eac5-7649-427a-a1e1-1a1e5e349e73 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42402024-12-14T04:48:59.573ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2075 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 86b5eac5-7649-427a-a1e1-1a1e5e349e73 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42412024-12-14T04:49:00.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7b1a3fb0-f239-4be3-bcf3-80e5ad97fcc1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42422024-12-14T04:49:00.594ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7b1a3fb0-f239-4be3-bcf3-80e5ad97fcc1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42432024-12-14T04:49:00.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2472 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7b1a3fb0-f239-4be3-bcf3-80e5ad97fcc1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42442024-12-14T04:49:01.616ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b1c94db-c696-43cd-b05e-6bb4692ed2cc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42452024-12-14T04:49:01.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b1c94db-c696-43cd-b05e-6bb4692ed2cc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42462024-12-14T04:49:01.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2671 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4b1c94db-c696-43cd-b05e-6bb4692ed2cc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42472024-12-14T04:49:02.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8ce9fb73-ab48-4398-8af7-4f6200302853 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42482024-12-14T04:49:02.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8ce9fb73-ab48-4398-8af7-4f6200302853 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42492024-12-14T04:49:02.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2902 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8ce9fb73-ab48-4398-8af7-4f6200302853 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42502024-12-14T04:49:02.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1489 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = a22b97c2-f816-4bdd-a770-116ac8f30bc8 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
42512024-12-14T04:49:03.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6364b1d2-03db-4c04-9dc5-5948551143f6 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42522024-12-14T04:49:03.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6364b1d2-03db-4c04-9dc5-5948551143f6 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42532024-12-14T04:49:03.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2807 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6364b1d2-03db-4c04-9dc5-5948551143f6 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42542024-12-14T04:49:04.683ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6e943ad6-48e4-4fc7-a541-f52cb577e7cd roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42552024-12-14T04:49:04.683ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6e943ad6-48e4-4fc7-a541-f52cb577e7cd resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42562024-12-14T04:49:04.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2003 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6e943ad6-48e4-4fc7-a541-f52cb577e7cd response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42572024-12-14T04:49:05.705ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e36e8a57-4534-4378-b757-656f787d1058 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42582024-12-14T04:49:05.705ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e36e8a57-4534-4378-b757-656f787d1058 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42592024-12-14T04:49:05.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2035 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e36e8a57-4534-4378-b757-656f787d1058 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42602024-12-14T04:49:06.793ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 99bd5b3d-9b51-488e-8a0a-d331b3de631b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42612024-12-14T04:49:06.793ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 99bd5b3d-9b51-488e-8a0a-d331b3de631b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42622024-12-14T04:49:06.795ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2890 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 99bd5b3d-9b51-488e-8a0a-d331b3de631b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
42632024-12-14T04:49:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
42642024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
42652024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
42662024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
42672024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42682024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
42692024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
42702024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42712024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
42722024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42732024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
42742024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
42752024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
42762024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42772024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
42782024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
42792024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42802024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
42812024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42822024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
42832024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
42842024-12-14T04:49:06.883ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
42852024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
42862024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
42872024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
42882024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
42892024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
42902024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
42912024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42922024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
42932024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42942024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
42952024-12-14T04:49:06.883ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
42962024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
42972024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42982024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
42992024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
43002024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
43012024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43022024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43032024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
43042024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43052024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
43062024-12-14T04:49:06.883ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832
43072024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.44\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832"}
43082024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
43092024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
43102024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
43112024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
43122024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
43132024-12-14T04:49:06.883ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 04:39:06.883323458 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
43142024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
43152024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
43162024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
43172024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43182024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
43192024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
43202024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
43212024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
43222024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
43232024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43242024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
43252024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
43262024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43272024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43282024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
43292024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43302024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
43312024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43322024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
43332024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
43342024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
43352024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
43362024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43372024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43382024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43392024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43402024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
43412024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
43422024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
43432024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43442024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
43452024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43462024-12-14T04:49:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ba1572d-df24-4bd2-be03-9a99edc867d8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1520, tv_nsec: 980337576 } })
43472024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43482024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
43492024-12-14T04:49:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43502024-12-14T04:49:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2224041a-a9fc-46d2-b9ed-ef98dc01f696.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("229c9d57-e318-46f6-8df9-9f602a126f12.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1520, tv_nsec: 980388837 } })
43512024-12-14T04:49:06.884ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
43522024-12-14T04:49:06.884ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.195471ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
43532024-12-14T04:49:06.884ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
43542024-12-14T04:49:06.884ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.074568ms iteration = 16 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
43552024-12-14T04:49:06.886ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.283726ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
43562024-12-14T04:49:06.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1520, tv_nsec: 983341365 } })
43572024-12-14T04:49:06.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43582024-12-14T04:49:06.887ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
43592024-12-14T04:49:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43602024-12-14T04:49:06.887ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.315732ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
43612024-12-14T04:49:06.887ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43622024-12-14T04:49:06.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43632024-12-14T04:49:06.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43642024-12-14T04:49:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
43652024-12-14T04:49:06.889ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43662024-12-14T04:49:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43672024-12-14T04:49:06.891ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
43682024-12-14T04:49:06.891ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
43692024-12-14T04:49:06.891ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 8.114991ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
43702024-12-14T04:49:06.891ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1520, tv_nsec: 988402766 } })
43712024-12-14T04:49:06.892ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
43722024-12-14T04:49:06.892ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43732024-12-14T04:49:06.892ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
43742024-12-14T04:49:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
43752024-12-14T04:49:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
43762024-12-14T04:49:06.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43772024-12-14T04:49:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43782024-12-14T04:49:06.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = cf8c22c3-9016-408c-82d2-ee86a3920cd5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
43792024-12-14T04:49:06.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = cf8c22c3-9016-408c-82d2-ee86a3920cd5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
43802024-12-14T04:49:06.894ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43812024-12-14T04:49:06.895ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43822024-12-14T04:49:06.896ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43832024-12-14T04:49:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
43842024-12-14T04:49:06.897ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
43852024-12-14T04:49:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
43862024-12-14T04:49:06.897ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43872024-12-14T04:49:06.897ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
43882024-12-14T04:49:06.897ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43892024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43902024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43912024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43922024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43932024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43942024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43952024-12-14T04:49:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43962024-12-14T04:49:06.899ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 04:39:06.883323458 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
43972024-12-14T04:49:06.899ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 16.10256ms iteration = 15 status = {"expiration":"2024-12-14T04:39:06.883323458Z","pruned":[]}
43982024-12-14T04:49:06.900ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43992024-12-14T04:49:06.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44002024-12-14T04:49:06.901ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44012024-12-14T04:49:06.901ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 18.116502ms iteration = 29 status = {"errors":[],"records_set_to_done":[]}
44022024-12-14T04:49:06.901ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44032024-12-14T04:49:06.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44042024-12-14T04:49:06.902ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44052024-12-14T04:49:06.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44062024-12-14T04:49:06.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
44072024-12-14T04:49:06.903ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44082024-12-14T04:49:06.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
44092024-12-14T04:49:06.905ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
44102024-12-14T04:49:06.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
44112024-12-14T04:49:06.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44122024-12-14T04:49:06.907ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1521, tv_nsec: 4389843 } })
44132024-12-14T04:49:06.908ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44142024-12-14T04:49:06.908ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
44152024-12-14T04:49:06.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
44162024-12-14T04:49:06.908ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44172024-12-14T04:49:06.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44182024-12-14T04:49:06.909ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
44192024-12-14T04:49:06.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44202024-12-14T04:49:06.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44212024-12-14T04:49:06.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 16331 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = cf8c22c3-9016-408c-82d2-ee86a3920cd5 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
44222024-12-14T04:49:06.911ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44232024-12-14T04:49:06.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44242024-12-14T04:49:06.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44252024-12-14T04:49:06.915ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44262024-12-14T04:49:06.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
44272024-12-14T04:49:06.920ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44282024-12-14T04:49:06.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
44292024-12-14T04:49:06.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
44302024-12-14T04:49:06.922ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44312024-12-14T04:49:06.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
44322024-12-14T04:49:06.934ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.348698ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
44332024-12-14T04:49:06.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44342024-12-14T04:49:06.934ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44352024-12-14T04:49:06.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44362024-12-14T04:49:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
44372024-12-14T04:49:06.938ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44382024-12-14T04:49:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44392024-12-14T04:49:06.938ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44402024-12-14T04:49:06.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
44412024-12-14T04:49:06.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44422024-12-14T04:49:06.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
44432024-12-14T04:49:06.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44442024-12-14T04:49:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44452024-12-14T04:49:06.948ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44462024-12-14T04:49:06.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44472024-12-14T04:49:06.955ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 72.051407ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44482024-12-14T04:49:06.955ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
44492024-12-14T04:49:06.955ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 72.1171ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
44502024-12-14T04:49:06.957ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
44512024-12-14T04:49:06.957ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
44522024-12-14T04:49:06.957ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
44532024-12-14T04:49:06.957ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44542024-12-14T04:49:06.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44552024-12-14T04:49:06.958ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44562024-12-14T04:49:06.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44572024-12-14T04:49:06.959ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44582024-12-14T04:49:06.959ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
44592024-12-14T04:49:06.959ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
44602024-12-14T04:49:06.959ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 76.76455ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
44612024-12-14T04:49:06.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44622024-12-14T04:49:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44632024-12-14T04:49:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
44642024-12-14T04:49:06.963ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44652024-12-14T04:49:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44662024-12-14T04:49:06.965ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44672024-12-14T04:49:06.965ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
44682024-12-14T04:49:06.965ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
44692024-12-14T04:49:06.965ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
44702024-12-14T04:49:06.965ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44712024-12-14T04:49:06.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44722024-12-14T04:49:06.966ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
44732024-12-14T04:49:06.966ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
44742024-12-14T04:49:06.966ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44752024-12-14T04:49:06.966ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44762024-12-14T04:49:06.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44772024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44782024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44792024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
44802024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44812024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44822024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44832024-12-14T04:49:06.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44842024-12-14T04:49:06.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44852024-12-14T04:49:06.970ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44862024-12-14T04:49:06.970ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
44872024-12-14T04:49:06.970ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44882024-12-14T04:49:06.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44892024-12-14T04:49:06.970ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 87.761146ms iteration = 29 status = 0
44902024-12-14T04:49:06.971ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44912024-12-14T04:49:06.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
44922024-12-14T04:49:06.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
44932024-12-14T04:49:06.972ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44942024-12-14T04:49:06.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
44952024-12-14T04:49:06.972ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44962024-12-14T04:49:06.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44972024-12-14T04:49:06.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44982024-12-14T04:49:06.973ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 90.148049ms iteration = 29 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
44992024-12-14T04:49:06.975ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
45002024-12-14T04:49:06.975ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 91.782681ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
45012024-12-14T04:49:06.977ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 94.094862ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
45022024-12-14T04:49:06.980ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45032024-12-14T04:49:06.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45042024-12-14T04:49:07.003ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45052024-12-14T04:49:07.003ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
45062024-12-14T04:49:07.003ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 120.566012ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
45072024-12-14T04:49:07.014ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
45082024-12-14T04:49:07.016ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
45092024-12-14T04:49:07.016ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
45102024-12-14T04:49:07.016ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b1cc6e0-e751-4957-8ea7-91db0a52a09d", "content-length": "26", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45112024-12-14T04:49:07.016ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
45122024-12-14T04:49:07.019ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
45132024-12-14T04:49:07.021ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
45142024-12-14T04:49:07.024ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45152024-12-14T04:49:07.025ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
45162024-12-14T04:49:07.025ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 141.804355ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
45172024-12-14T04:49:07.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45182024-12-14T04:49:07.035ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 152.583806ms iteration = 33 status = {"physical_disks_added":0}
45192024-12-14T04:49:07.046ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
45202024-12-14T04:49:07.046ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
45212024-12-14T04:49:07.046ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "73007083-eed8-4eab-b753-c377047728d2", "content-length": "26", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45222024-12-14T04:49:07.046ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
45232024-12-14T04:49:07.049ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
45242024-12-14T04:49:07.063ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
45252024-12-14T04:49:07.063ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
45262024-12-14T04:49:07.063ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7631827-06a7-48d8-8cc6-70207679c41d", "content-length": "26", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45272024-12-14T04:49:07.064ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
45282024-12-14T04:49:07.065ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
45292024-12-14T04:49:07.198ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
45302024-12-14T04:49:07.198ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45312024-12-14T04:49:07.199ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce100f86-8611-4f4d-a431-3e6b4c597db8", "content-length": "372", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45322024-12-14T04:49:07.199ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45332024-12-14T04:49:07.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45342024-12-14T04:49:07.200ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45352024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
45362024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45372024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
45382024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 318.980182ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
45392024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
45402024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
45412024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
45422024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 04:35:18.025163 UTC dns_group = internal generation = 1
45432024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 319.238308ms iteration = 16 status = {"generation":1}
45442024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
45452024-12-14T04:49:07.202ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "82e52ddd-7b2c-4ed9-9b75-c24735dd0f18", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
45462024-12-14T04:49:07.202ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
45472024-12-14T04:49:07.204ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "da05109e-dedc-4948-bbd6-364bfd125754", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
45482024-12-14T04:49:07.204ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
45492024-12-14T04:49:07.206ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
45502024-12-14T04:49:07.206ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 322.859012ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
45512024-12-14T04:49:07.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45522024-12-14T04:49:07.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45532024-12-14T04:49:07.210ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45542024-12-14T04:49:07.212ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45552024-12-14T04:49:07.212ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45562024-12-14T04:49:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45572024-12-14T04:49:07.222ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 339.793613ms iteration = 29 status = {}
45582024-12-14T04:49:07.229ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
45592024-12-14T04:49:07.229ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eeff836-cdba-4688-97b5-a97715e69640", "content-length": "2", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45602024-12-14T04:49:07.232ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
45612024-12-14T04:49:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
45622024-12-14T04:49:07.234ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 351.872688ms iteration = 29 status = null
45632024-12-14T04:49:07.237ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
45642024-12-14T04:49:07.237ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 0 version = 2
45652024-12-14T04:49:07.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45662024-12-14T04:49:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45672024-12-14T04:49:07.239ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T04:35:03.711237Z, time_modified: 2024-12-14T04:35:03.711237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }) }
45682024-12-14T04:49:07.239ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 356.418077ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]}},"default_endpoint":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
45692024-12-14T04:49:07.248ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45702024-12-14T04:49:07.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45712024-12-14T04:49:07.250ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45722024-12-14T04:49:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45732024-12-14T04:49:07.253ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
45742024-12-14T04:49:07.253ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
45752024-12-14T04:49:07.253ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1e61efbf-bb34-4e83-8f16-b45c823ba50d", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45762024-12-14T04:49:07.253ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45772024-12-14T04:49:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45782024-12-14T04:49:07.253ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
45792024-12-14T04:49:07.254ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "fd866005-fce7-419f-8667-f54461cdcfdb", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45802024-12-14T04:49:07.254ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45812024-12-14T04:49:07.255ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7ed501e2-daf1-4001-b87a-27234f81a335", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45822024-12-14T04:49:07.255ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45832024-12-14T04:49:07.256ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45842024-12-14T04:49:07.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45852024-12-14T04:49:07.257ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "1872453c-ed32-47bd-a3bc-f7bc12ccb632", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45862024-12-14T04:49:07.257ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45872024-12-14T04:49:07.259ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45882024-12-14T04:49:07.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45892024-12-14T04:49:07.259ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
45902024-12-14T04:49:07.259ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 376.418128ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
45912024-12-14T04:49:07.262ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
45922024-12-14T04:49:07.262ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45932024-12-14T04:49:07.262ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "88a1d129-3cde-4827-8203-5cbc1235cf04", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
45942024-12-14T04:49:07.265ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 381.782098ms iteration = 31 status = {}
45952024-12-14T04:49:07.265ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45962024-12-14T04:49:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
45972024-12-14T04:49:07.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45982024-12-14T04:49:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45992024-12-14T04:49:07.293ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
46002024-12-14T04:49:07.293ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd0cbde0-381c-4726-acd0-09d5f7703c39", "content-length": "691", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
46012024-12-14T04:49:07.293ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
46022024-12-14T04:49:07.293ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
46032024-12-14T04:49:07.293ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
46042024-12-14T04:49:07.294ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 411.684877ms iteration = 30 status = {}
46052024-12-14T04:49:07.295ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 1 version = 2
46062024-12-14T04:49:07.295ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
46072024-12-14T04:49:07.295ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 04:35:19.721992 UTC dns_group = external generation = 2
46082024-12-14T04:49:07.295ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 411.892763ms iteration = 16 status = {"generation":2}
46092024-12-14T04:49:07.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46102024-12-14T04:49:07.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46112024-12-14T04:49:07.298ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
46122024-12-14T04:49:07.298ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46132024-12-14T04:49:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46142024-12-14T04:49:07.308ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
46152024-12-14T04:49:07.308ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "29f68445-fd6b-4bd0-865d-c083b4169e14", "content-length": "134", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
46162024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
46172024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
46182024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
46192024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
46202024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46212024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46222024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46232024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
46242024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46252024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
46262024-12-14T04:49:07.308ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
46272024-12-14T04:49:07.308ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46282024-12-14T04:49:07.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46292024-12-14T04:49:07.311ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
46302024-12-14T04:49:07.311ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6c6d45d-4982-4c40-95ab-484b738941b2", "content-length": "2", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
46312024-12-14T04:49:07.311ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
46322024-12-14T04:49:07.311ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb0cd395-bc8e-400f-a6f0-ca1d69a43264", "content-length": "52", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
46332024-12-14T04:49:07.311ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
46342024-12-14T04:49:07.311ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
46352024-12-14T04:49:07.408ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c8dd346b-6a91-4083-9f0e-59d7da721282", "date": "Sat, 14 Dec 2024 04:49:06 GMT"} })
46362024-12-14T04:49:07.408ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
46372024-12-14T04:49:07.408ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
46382024-12-14T04:49:07.408ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb69a5b2-fb0c-494b-b587-fa43c646199b", "content-length": "682", "date": "Sat, 14 Dec 2024 04:49:07 GMT"} })
46392024-12-14T04:49:07.408ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46402024-12-14T04:49:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46412024-12-14T04:49:07.412ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46422024-12-14T04:49:07.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46432024-12-14T04:49:07.415ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46442024-12-14T04:49:07.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46452024-12-14T04:49:07.417ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46462024-12-14T04:49:07.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46472024-12-14T04:49:07.425ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 542.107726ms iteration = 29 status = {}
46482024-12-14T04:49:07.815ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c7086d68-d321-49ab-acd7-36579e7e545f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46492024-12-14T04:49:07.815ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c7086d68-d321-49ab-acd7-36579e7e545f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46502024-12-14T04:49:07.817ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2254 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c7086d68-d321-49ab-acd7-36579e7e545f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46512024-12-14T04:49:08.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e0866302-4041-4fe2-a2f3-26414675eb9d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46522024-12-14T04:49:08.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e0866302-4041-4fe2-a2f3-26414675eb9d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46532024-12-14T04:49:08.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2661 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e0866302-4041-4fe2-a2f3-26414675eb9d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46542024-12-14T04:49:09.859ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 88f12c01-1051-4260-9cac-627dca31a05e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46552024-12-14T04:49:09.859ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 88f12c01-1051-4260-9cac-627dca31a05e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46562024-12-14T04:49:09.861ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2355 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 88f12c01-1051-4260-9cac-627dca31a05e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46572024-12-14T04:49:10.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b356d0c1-45d8-4a10-a35d-57040ab47456 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46582024-12-14T04:49:10.881ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b356d0c1-45d8-4a10-a35d-57040ab47456 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46592024-12-14T04:49:10.883ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2560 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b356d0c1-45d8-4a10-a35d-57040ab47456 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46602024-12-14T04:49:11.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5a26b410-d86b-4e05-9067-c2183d9cb110 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46612024-12-14T04:49:11.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5a26b410-d86b-4e05-9067-c2183d9cb110 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46622024-12-14T04:49:11.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2448 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5a26b410-d86b-4e05-9067-c2183d9cb110 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46632024-12-14T04:49:12.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1312 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = e956b1c1-23c0-4320-84aa-a39e041be121 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
46642024-12-14T04:49:12.926ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 620ea622-f09f-47de-8fc5-31abbac4438f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46652024-12-14T04:49:12.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 620ea622-f09f-47de-8fc5-31abbac4438f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46662024-12-14T04:49:12.928ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2319 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 620ea622-f09f-47de-8fc5-31abbac4438f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46672024-12-14T04:49:13.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b7f30fcf-9d4c-4c63-ba4a-cf44a3aff29e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46682024-12-14T04:49:13.949ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b7f30fcf-9d4c-4c63-ba4a-cf44a3aff29e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46692024-12-14T04:49:13.951ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2522 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b7f30fcf-9d4c-4c63-ba4a-cf44a3aff29e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46702024-12-14T04:49:14.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ae6ed237-46d5-41e8-b9f3-baeaed064f8a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46712024-12-14T04:49:14.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ae6ed237-46d5-41e8-b9f3-baeaed064f8a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46722024-12-14T04:49:14.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2123 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ae6ed237-46d5-41e8-b9f3-baeaed064f8a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46732024-12-14T04:49:15.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6429237d-93b5-4b00-96cc-edf9e4524304 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46742024-12-14T04:49:15.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6429237d-93b5-4b00-96cc-edf9e4524304 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46752024-12-14T04:49:15.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2416 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6429237d-93b5-4b00-96cc-edf9e4524304 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46762024-12-14T04:49:16.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
46772024-12-14T04:49:16.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46782024-12-14T04:49:16.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46792024-12-14T04:49:16.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
46802024-12-14T04:49:16.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46812024-12-14T04:49:16.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46822024-12-14T04:49:16.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46832024-12-14T04:49:16.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46842024-12-14T04:49:16.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46852024-12-14T04:49:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46862024-12-14T04:49:16.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46872024-12-14T04:49:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46882024-12-14T04:49:16.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46892024-12-14T04:49:16.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46902024-12-14T04:49:16.913ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.4666ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46912024-12-14T04:49:16.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46922024-12-14T04:49:16.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
46932024-12-14T04:49:16.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46942024-12-14T04:49:16.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46952024-12-14T04:49:16.934ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
46962024-12-14T04:49:16.934ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.697211ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
46972024-12-14T04:49:17.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 484e7cc1-a34a-4ee8-a24c-8ad9ff83697c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46982024-12-14T04:49:17.013ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 484e7cc1-a34a-4ee8-a24c-8ad9ff83697c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
46992024-12-14T04:49:17.015ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2102 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 484e7cc1-a34a-4ee8-a24c-8ad9ff83697c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47002024-12-14T04:49:17.771ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 5f2ee44e-b61b-4dc3-acad-40ab7751b650 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
47012024-12-14T04:49:17.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 5f2ee44e-b61b-4dc3-acad-40ab7751b650 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
47022024-12-14T04:49:17.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3550 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 5f2ee44e-b61b-4dc3-acad-40ab7751b650 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
47032024-12-14T04:49:17.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 1aeaf6a2-3223-4fef-be99-10eb95edd0ca roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
47042024-12-14T04:49:17.775ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 1aeaf6a2-3223-4fef-be99-10eb95edd0ca resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
47052024-12-14T04:49:17.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2168 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 1aeaf6a2-3223-4fef-be99-10eb95edd0ca response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
47062024-12-14T04:49:18.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b901119c-e07a-4a8d-922d-af48945c4659 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47072024-12-14T04:49:18.037ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b901119c-e07a-4a8d-922d-af48945c4659 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47082024-12-14T04:49:18.039ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2493 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b901119c-e07a-4a8d-922d-af48945c4659 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47092024-12-14T04:49:19.059ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 156efa36-a62a-4ef7-aebb-37a4aa1cb3cc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47102024-12-14T04:49:19.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 156efa36-a62a-4ef7-aebb-37a4aa1cb3cc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47112024-12-14T04:49:19.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2381 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 156efa36-a62a-4ef7-aebb-37a4aa1cb3cc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47122024-12-14T04:49:20.079ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5888f77b-9662-447f-ae2d-e91d9464b923 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47132024-12-14T04:49:20.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5888f77b-9662-447f-ae2d-e91d9464b923 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47142024-12-14T04:49:20.081ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2389 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5888f77b-9662-447f-ae2d-e91d9464b923 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47152024-12-14T04:49:21.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8c97888d-82fd-423e-b687-3e81f625c45d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47162024-12-14T04:49:21.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8c97888d-82fd-423e-b687-3e81f625c45d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47172024-12-14T04:49:21.103ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2643 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8c97888d-82fd-423e-b687-3e81f625c45d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47182024-12-14T04:49:22.123ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9932d4c6-7f3d-40c4-a734-8e45331bae64 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47192024-12-14T04:49:22.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9932d4c6-7f3d-40c4-a734-8e45331bae64 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47202024-12-14T04:49:22.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9932d4c6-7f3d-40c4-a734-8e45331bae64 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47212024-12-14T04:49:22.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1443 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = c9009e5b-55f9-430f-9145-9dd52e84bab3 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
47222024-12-14T04:49:23.145ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ce30591f-ee52-48be-8952-04fffcf08835 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47232024-12-14T04:49:23.146ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ce30591f-ee52-48be-8952-04fffcf08835 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47242024-12-14T04:49:23.148ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2470 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ce30591f-ee52-48be-8952-04fffcf08835 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47252024-12-14T04:49:24.168ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7ef16378-aed0-4738-b1a1-77d2a4d3fc23 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47262024-12-14T04:49:24.168ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7ef16378-aed0-4738-b1a1-77d2a4d3fc23 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47272024-12-14T04:49:24.170ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2084 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7ef16378-aed0-4738-b1a1-77d2a4d3fc23 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47282024-12-14T04:49:25.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 96376065-8fb9-4e32-abd9-4f23b4df0a7c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47292024-12-14T04:49:25.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 96376065-8fb9-4e32-abd9-4f23b4df0a7c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47302024-12-14T04:49:25.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2852 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 96376065-8fb9-4e32-abd9-4f23b4df0a7c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47312024-12-14T04:49:26.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 161d3582-4cdc-4b47-93fc-c2b7f9a74d89 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47322024-12-14T04:49:26.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 161d3582-4cdc-4b47-93fc-c2b7f9a74d89 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47332024-12-14T04:49:26.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2298 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 161d3582-4cdc-4b47-93fc-c2b7f9a74d89 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47342024-12-14T04:49:26.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
47352024-12-14T04:49:26.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47362024-12-14T04:49:26.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47372024-12-14T04:49:26.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
47382024-12-14T04:49:26.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47392024-12-14T04:49:26.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47402024-12-14T04:49:26.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47412024-12-14T04:49:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47422024-12-14T04:49:26.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47432024-12-14T04:49:26.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47442024-12-14T04:49:26.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47452024-12-14T04:49:26.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47462024-12-14T04:49:26.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47472024-12-14T04:49:26.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
47482024-12-14T04:49:26.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47492024-12-14T04:49:26.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47502024-12-14T04:49:26.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
47512024-12-14T04:49:26.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.55258ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
47522024-12-14T04:49:26.919ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47532024-12-14T04:49:26.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47542024-12-14T04:49:26.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.851652ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47552024-12-14T04:49:27.235ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = deb7f63a-6e67-4758-8324-639ae9f34932 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47562024-12-14T04:49:27.236ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = deb7f63a-6e67-4758-8324-639ae9f34932 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47572024-12-14T04:49:27.237ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2220 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = deb7f63a-6e67-4758-8324-639ae9f34932 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47582024-12-14T04:49:28.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 01e7f46a-12b1-42da-88f8-2838bd32c490 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47592024-12-14T04:49:28.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 01e7f46a-12b1-42da-88f8-2838bd32c490 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47602024-12-14T04:49:28.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2673 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 01e7f46a-12b1-42da-88f8-2838bd32c490 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47612024-12-14T04:49:29.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9a0a8edf-faec-4075-8e82-c642b4e95c24 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47622024-12-14T04:49:29.281ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9a0a8edf-faec-4075-8e82-c642b4e95c24 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47632024-12-14T04:49:29.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3151 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9a0a8edf-faec-4075-8e82-c642b4e95c24 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47642024-12-14T04:49:30.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 137f7837-5e2c-4476-ad3b-1da36df6b0a8 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47652024-12-14T04:49:30.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 137f7837-5e2c-4476-ad3b-1da36df6b0a8 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47662024-12-14T04:49:30.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2345 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 137f7837-5e2c-4476-ad3b-1da36df6b0a8 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47672024-12-14T04:49:31.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e491e4e9-7352-42c0-87e5-519953037db0 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47682024-12-14T04:49:31.327ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e491e4e9-7352-42c0-87e5-519953037db0 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47692024-12-14T04:49:31.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2242 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e491e4e9-7352-42c0-87e5-519953037db0 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47702024-12-14T04:49:32.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b90ffaed-31fd-4fae-ac0d-c2133d33d309 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47712024-12-14T04:49:32.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b90ffaed-31fd-4fae-ac0d-c2133d33d309 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47722024-12-14T04:49:32.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2272 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b90ffaed-31fd-4fae-ac0d-c2133d33d309 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47732024-12-14T04:49:32.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 3033799f-51ab-46ea-b0c4-1a1cae9847be roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
47742024-12-14T04:49:32.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 3033799f-51ab-46ea-b0c4-1a1cae9847be resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
47752024-12-14T04:49:32.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3456 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 3033799f-51ab-46ea-b0c4-1a1cae9847be response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
47762024-12-14T04:49:32.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 52a9e5f7-1b0e-4626-853b-5b9104006865 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
47772024-12-14T04:49:32.774ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 52a9e5f7-1b0e-4626-853b-5b9104006865 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
47782024-12-14T04:49:32.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2663 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 52a9e5f7-1b0e-4626-853b-5b9104006865 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
47792024-12-14T04:49:32.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1462 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 2b555698-ffce-42ba-971f-b11d9af6c899 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
47802024-12-14T04:49:33.371ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34cc3d48-ca42-4a32-ac68-28ff94ecea61 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47812024-12-14T04:49:33.371ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34cc3d48-ca42-4a32-ac68-28ff94ecea61 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47822024-12-14T04:49:33.373ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2305 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34cc3d48-ca42-4a32-ac68-28ff94ecea61 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47832024-12-14T04:49:34.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 865c931a-710e-4e03-bd0f-d712e9de55f5 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47842024-12-14T04:49:34.395ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 865c931a-710e-4e03-bd0f-d712e9de55f5 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47852024-12-14T04:49:34.396ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2004 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 865c931a-710e-4e03-bd0f-d712e9de55f5 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47862024-12-14T04:49:35.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ce3e789a-4993-452f-a981-8f6b7551a3ea roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47872024-12-14T04:49:35.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ce3e789a-4993-452f-a981-8f6b7551a3ea resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47882024-12-14T04:49:35.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2229 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ce3e789a-4993-452f-a981-8f6b7551a3ea response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47892024-12-14T04:49:36.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c9da6d35-ef93-4bc5-bc8c-1ae1190cc825 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47902024-12-14T04:49:36.508ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c9da6d35-ef93-4bc5-bc8c-1ae1190cc825 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47912024-12-14T04:49:36.510ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2376 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c9da6d35-ef93-4bc5-bc8c-1ae1190cc825 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
47922024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
47932024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
47942024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
47952024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47962024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
47972024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
47982024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
47992024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
48002024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
48012024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
48022024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
48032024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48042024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
48052024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48062024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48072024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
48082024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48092024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
48102024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
48112024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48122024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48132024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
48142024-12-14T04:49:36.882ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
48152024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
48162024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48172024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48182024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
48192024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
48202024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
48212024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48222024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48232024-12-14T04:49:36.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
48242024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
48252024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48262024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
48272024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
48282024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48292024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48302024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48312024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48322024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48332024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48342024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48352024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48362024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48372024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
48382024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
48392024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48402024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48412024-12-14T04:49:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48422024-12-14T04:49:36.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48432024-12-14T04:49:36.885ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.795543ms iteration = 30 status = {"errors":[],"records_set_to_done":[]}
48442024-12-14T04:49:36.886ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.86392ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
48452024-12-14T04:49:36.887ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1550, tv_nsec: 983654933 } })
48462024-12-14T04:49:36.887ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48472024-12-14T04:49:36.887ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
48482024-12-14T04:49:36.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48492024-12-14T04:49:36.888ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48502024-12-14T04:49:36.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48512024-12-14T04:49:36.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48522024-12-14T04:49:36.889ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1550, tv_nsec: 985961202 } })
48532024-12-14T04:49:36.889ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
48542024-12-14T04:49:36.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
48552024-12-14T04:49:36.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48562024-12-14T04:49:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48572024-12-14T04:49:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48582024-12-14T04:49:36.890ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48592024-12-14T04:49:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48602024-12-14T04:49:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
48612024-12-14T04:49:36.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48622024-12-14T04:49:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48632024-12-14T04:49:36.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48642024-12-14T04:49:36.894ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48652024-12-14T04:49:36.894ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48662024-12-14T04:49:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48672024-12-14T04:49:36.896ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1550, tv_nsec: 992809211 } })
48682024-12-14T04:49:36.896ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
48692024-12-14T04:49:36.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
48702024-12-14T04:49:36.896ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48712024-12-14T04:49:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48722024-12-14T04:49:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48732024-12-14T04:49:36.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48742024-12-14T04:49:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48752024-12-14T04:49:36.899ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48762024-12-14T04:49:36.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48772024-12-14T04:49:36.900ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48782024-12-14T04:49:36.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48792024-12-14T04:49:36.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48802024-12-14T04:49:36.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48812024-12-14T04:49:36.901ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48822024-12-14T04:49:36.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48832024-12-14T04:49:36.903ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48842024-12-14T04:49:36.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48852024-12-14T04:49:36.914ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48862024-12-14T04:49:36.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48872024-12-14T04:49:36.917ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48882024-12-14T04:49:36.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48892024-12-14T04:49:36.925ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 42.619341ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
48902024-12-14T04:49:36.925ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
48912024-12-14T04:49:36.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
48922024-12-14T04:49:36.926ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48932024-12-14T04:49:36.926ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48942024-12-14T04:49:36.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48952024-12-14T04:49:36.927ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 45.144446ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
48962024-12-14T04:49:36.927ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48972024-12-14T04:49:36.927ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48982024-12-14T04:49:36.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48992024-12-14T04:49:36.928ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49002024-12-14T04:49:36.929ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49012024-12-14T04:49:36.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49022024-12-14T04:49:36.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49032024-12-14T04:49:36.930ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49042024-12-14T04:49:36.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49052024-12-14T04:49:36.930ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49062024-12-14T04:49:36.931ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 49.322835ms iteration = 30 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
49072024-12-14T04:49:36.933ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
49082024-12-14T04:49:36.933ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49092024-12-14T04:49:36.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49102024-12-14T04:49:36.934ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 52.305583ms iteration = 30 status = 0
49112024-12-14T04:49:36.938ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49122024-12-14T04:49:36.938ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
49132024-12-14T04:49:36.938ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49142024-12-14T04:49:36.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49152024-12-14T04:49:36.939ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.506452ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49162024-12-14T04:49:36.941ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49172024-12-14T04:49:36.941ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
49182024-12-14T04:49:36.941ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49192024-12-14T04:49:36.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 0c70b327-4b77-448e-bc09-26f9046070b7 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
49202024-12-14T04:49:36.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49212024-12-14T04:49:36.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 0c70b327-4b77-448e-bc09-26f9046070b7 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
49222024-12-14T04:49:36.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49232024-12-14T04:49:36.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3670 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 0c70b327-4b77-448e-bc09-26f9046070b7 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
49242024-12-14T04:49:36.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
49252024-12-14T04:49:36.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49262024-12-14T04:49:36.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49272024-12-14T04:49:36.947ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
49282024-12-14T04:49:36.947ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49292024-12-14T04:49:36.947ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdbb377a-26cb-4aa2-bdfd-2ac1bd6d2b7e", "content-length": "26", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49302024-12-14T04:49:36.947ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49312024-12-14T04:49:36.948ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
49322024-12-14T04:49:36.948ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49332024-12-14T04:49:36.948ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38fdf2dd-8dda-4ca7-9695-c86b39ddd1d1", "content-length": "26", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49342024-12-14T04:49:36.948ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49352024-12-14T04:49:36.949ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
49362024-12-14T04:49:36.949ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
49372024-12-14T04:49:36.952ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 70.049765ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
49382024-12-14T04:49:36.955ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
49392024-12-14T04:49:36.955ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 71.288618ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
49402024-12-14T04:49:36.956ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
49412024-12-14T04:49:36.956ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 73.176297ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
49422024-12-14T04:49:36.970ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49432024-12-14T04:49:36.970ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
49442024-12-14T04:49:36.970ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 87.414057ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
49452024-12-14T04:49:36.987ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49462024-12-14T04:49:36.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49472024-12-14T04:49:36.991ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
49482024-12-14T04:49:36.991ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49492024-12-14T04:49:36.991ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a4f8661-1613-4491-938f-49c7739859c8", "content-length": "26", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49502024-12-14T04:49:36.991ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49512024-12-14T04:49:36.992ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
49522024-12-14T04:49:36.995ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49532024-12-14T04:49:36.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49542024-12-14T04:49:36.996ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49552024-12-14T04:49:36.998ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49562024-12-14T04:49:36.998ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49572024-12-14T04:49:36.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49582024-12-14T04:49:36.999ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49592024-12-14T04:49:37.001ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49602024-12-14T04:49:37.001ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49612024-12-14T04:49:37.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49622024-12-14T04:49:37.004ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 121.960967ms iteration = 34 status = {"physical_disks_added":0}
49632024-12-14T04:49:37.029ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
49642024-12-14T04:49:37.029ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49652024-12-14T04:49:37.029ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6bd24d5-e9ab-4313-a99e-7271e518c12e", "content-length": "372", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49662024-12-14T04:49:37.029ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49672024-12-14T04:49:37.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49682024-12-14T04:49:37.064ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 182.28765ms iteration = 30 status = {}
49692024-12-14T04:49:37.071ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
49702024-12-14T04:49:37.071ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
49712024-12-14T04:49:37.071ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1b4598b1-199e-452f-9a87-91639082e805", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49722024-12-14T04:49:37.071ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49732024-12-14T04:49:37.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49742024-12-14T04:49:37.073ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49752024-12-14T04:49:37.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49762024-12-14T04:49:37.077ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49772024-12-14T04:49:37.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49782024-12-14T04:49:37.080ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
49792024-12-14T04:49:37.080ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "c34bd517-3c87-49e8-b9f1-93f1309854e2", "content-length": "2", "date": "Sat, 14 Dec 2024 04:49:37 GMT"} })
49802024-12-14T04:49:37.082ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
49812024-12-14T04:49:37.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
49822024-12-14T04:49:37.098ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
49832024-12-14T04:49:37.098ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0aacfdf-f4fd-47f0-bf55-9ebe855d9d76", "content-length": "691", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49842024-12-14T04:49:37.098ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
49852024-12-14T04:49:37.098ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
49862024-12-14T04:49:37.098ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
49872024-12-14T04:49:37.100ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 217.97414ms iteration = 30 status = null
49882024-12-14T04:49:37.101ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 218.329009ms iteration = 31 status = {}
49892024-12-14T04:49:37.101ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
49902024-12-14T04:49:37.101ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49912024-12-14T04:49:37.101ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c1ca60e8-e578-4f71-a100-d0a7559c6adc", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
49922024-12-14T04:49:37.103ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 220.496275ms iteration = 32 status = {}
49932024-12-14T04:49:37.156ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49942024-12-14T04:49:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49952024-12-14T04:49:37.158ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
49962024-12-14T04:49:37.158ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49972024-12-14T04:49:37.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49982024-12-14T04:49:37.167ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
49992024-12-14T04:49:37.167ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff592e4b-520e-4e55-b9b0-4f622d5fe410", "content-length": "134", "date": "Sat, 14 Dec 2024 04:49:37 GMT"} })
50002024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
50012024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
50022024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
50032024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
50042024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50052024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50062024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50072024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50082024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50092024-12-14T04:49:37.167ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50102024-12-14T04:49:37.168ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
50112024-12-14T04:49:37.168ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50122024-12-14T04:49:37.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50132024-12-14T04:49:37.170ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
50142024-12-14T04:49:37.170ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd9658f3-3ea4-4393-8da8-2c383104cffa", "content-length": "2", "date": "Sat, 14 Dec 2024 04:49:37 GMT"} })
50152024-12-14T04:49:37.170ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
50162024-12-14T04:49:37.170ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "72c1112d-7b45-4fc9-8eb6-55a780abf9e0", "content-length": "52", "date": "Sat, 14 Dec 2024 04:49:37 GMT"} })
50172024-12-14T04:49:37.170ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
50182024-12-14T04:49:37.170ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
50192024-12-14T04:49:37.268ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2e99ddfc-73b8-48d1-82c2-46995f26aee4", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
50202024-12-14T04:49:37.268ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
50212024-12-14T04:49:37.268ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
50222024-12-14T04:49:37.268ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ebdaca5-769a-4f0d-bc94-ec070344eb55", "content-length": "682", "date": "Sat, 14 Dec 2024 04:49:36 GMT"} })
50232024-12-14T04:49:37.268ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50242024-12-14T04:49:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50252024-12-14T04:49:37.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50262024-12-14T04:49:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50272024-12-14T04:49:37.276ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50282024-12-14T04:49:37.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50292024-12-14T04:49:37.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50302024-12-14T04:49:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50312024-12-14T04:49:37.286ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 403.604997ms iteration = 30 status = {}
50322024-12-14T04:49:37.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ec06070f-8624-42dc-9bff-930fc2357508 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50332024-12-14T04:49:37.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ec06070f-8624-42dc-9bff-930fc2357508 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50342024-12-14T04:49:37.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1607 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ec06070f-8624-42dc-9bff-930fc2357508 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50352024-12-14T04:49:38.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a4cf55a1-f7a0-4eb6-8677-663459909601 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50362024-12-14T04:49:38.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a4cf55a1-f7a0-4eb6-8677-663459909601 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50372024-12-14T04:49:38.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2352 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a4cf55a1-f7a0-4eb6-8677-663459909601 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50382024-12-14T04:49:39.574ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9beabf52-85d8-43c0-a85b-40a326a60cf2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50392024-12-14T04:49:39.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9beabf52-85d8-43c0-a85b-40a326a60cf2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50402024-12-14T04:49:39.576ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2439 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9beabf52-85d8-43c0-a85b-40a326a60cf2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50412024-12-14T04:49:40.596ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b00f359-0082-4f8b-9b1c-6ac934af5548 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50422024-12-14T04:49:40.596ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b00f359-0082-4f8b-9b1c-6ac934af5548 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50432024-12-14T04:49:40.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2248 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8b00f359-0082-4f8b-9b1c-6ac934af5548 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50442024-12-14T04:49:41.618ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 414ae9fe-dd1b-4997-bb98-41019bbecfbe roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50452024-12-14T04:49:41.618ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 414ae9fe-dd1b-4997-bb98-41019bbecfbe resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50462024-12-14T04:49:41.620ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2402 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 414ae9fe-dd1b-4997-bb98-41019bbecfbe response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50472024-12-14T04:49:42.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f8bfb112-bc74-4fcb-9b42-0074080ddc39 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50482024-12-14T04:49:42.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f8bfb112-bc74-4fcb-9b42-0074080ddc39 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50492024-12-14T04:49:42.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1909 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f8bfb112-bc74-4fcb-9b42-0074080ddc39 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50502024-12-14T04:49:42.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1282 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 426164a1-e777-4799-bf98-439c054db002 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
50512024-12-14T04:49:43.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 240e78b2-0ffa-4ff9-ac7b-6e2976634950 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50522024-12-14T04:49:43.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 240e78b2-0ffa-4ff9-ac7b-6e2976634950 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50532024-12-14T04:49:43.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2470 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 240e78b2-0ffa-4ff9-ac7b-6e2976634950 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50542024-12-14T04:49:44.683ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4bf6f666-6c60-484e-8704-0c3463ed6cbf roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50552024-12-14T04:49:44.684ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4bf6f666-6c60-484e-8704-0c3463ed6cbf resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50562024-12-14T04:49:44.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2301 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4bf6f666-6c60-484e-8704-0c3463ed6cbf response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50572024-12-14T04:49:45.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1fe23898-a896-4c4e-94e1-7a59fe8c7746 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50582024-12-14T04:49:45.704ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1fe23898-a896-4c4e-94e1-7a59fe8c7746 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50592024-12-14T04:49:45.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2512 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1fe23898-a896-4c4e-94e1-7a59fe8c7746 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50602024-12-14T04:49:46.725ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4f83950d-7b74-497f-b91a-0109211cbba3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50612024-12-14T04:49:46.725ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4f83950d-7b74-497f-b91a-0109211cbba3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50622024-12-14T04:49:46.727ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2095 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4f83950d-7b74-497f-b91a-0109211cbba3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50632024-12-14T04:49:46.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
50642024-12-14T04:49:46.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50652024-12-14T04:49:46.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50662024-12-14T04:49:46.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
50672024-12-14T04:49:46.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50682024-12-14T04:49:46.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50692024-12-14T04:49:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50702024-12-14T04:49:46.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50712024-12-14T04:49:46.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50722024-12-14T04:49:46.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.437726ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50732024-12-14T04:49:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50742024-12-14T04:49:46.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50752024-12-14T04:49:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50762024-12-14T04:49:46.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50772024-12-14T04:49:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50782024-12-14T04:49:46.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50792024-12-14T04:49:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
50802024-12-14T04:49:46.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50812024-12-14T04:49:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50822024-12-14T04:49:46.905ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
50832024-12-14T04:49:46.905ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.600163ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
50842024-12-14T04:49:47.747ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7478df86-c8f0-41de-b21c-7f0654150280 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50852024-12-14T04:49:47.748ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7478df86-c8f0-41de-b21c-7f0654150280 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50862024-12-14T04:49:47.750ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2745 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7478df86-c8f0-41de-b21c-7f0654150280 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50872024-12-14T04:49:47.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c78fd291-f722-4f59-8454-5b38a7c00b23 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
50882024-12-14T04:49:47.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c78fd291-f722-4f59-8454-5b38a7c00b23 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
50892024-12-14T04:49:47.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2882 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c78fd291-f722-4f59-8454-5b38a7c00b23 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
50902024-12-14T04:49:47.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c6d40554-7071-4531-9f4c-34cab61f6d9b roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
50912024-12-14T04:49:47.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c6d40554-7071-4531-9f4c-34cab61f6d9b resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
50922024-12-14T04:49:47.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1904 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c6d40554-7071-4531-9f4c-34cab61f6d9b response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
50932024-12-14T04:49:48.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3402b7fd-d281-4cea-9087-e874b2557b59 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50942024-12-14T04:49:48.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3402b7fd-d281-4cea-9087-e874b2557b59 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50952024-12-14T04:49:48.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2373 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3402b7fd-d281-4cea-9087-e874b2557b59 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50962024-12-14T04:49:49.792ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ffd560e8-d0ba-4e96-82b5-cda32f24bd29 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50972024-12-14T04:49:49.793ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ffd560e8-d0ba-4e96-82b5-cda32f24bd29 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50982024-12-14T04:49:49.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ffd560e8-d0ba-4e96-82b5-cda32f24bd29 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
50992024-12-14T04:49:50.814ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ae7e069c-fc69-4a14-8d14-03240bca048f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51002024-12-14T04:49:50.814ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ae7e069c-fc69-4a14-8d14-03240bca048f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51012024-12-14T04:49:50.816ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2347 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ae7e069c-fc69-4a14-8d14-03240bca048f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51022024-12-14T04:49:51.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e151f477-a735-4acd-9d50-e39f0a5c7709 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51032024-12-14T04:49:51.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e151f477-a735-4acd-9d50-e39f0a5c7709 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51042024-12-14T04:49:51.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2312 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e151f477-a735-4acd-9d50-e39f0a5c7709 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51052024-12-14T04:49:52.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1326 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 7627d0db-a449-4afd-8f5c-9499ed9a0ccf response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
51062024-12-14T04:49:52.859ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 47a388f0-8abe-4c2e-958a-08187f6c9be6 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51072024-12-14T04:49:52.859ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 47a388f0-8abe-4c2e-958a-08187f6c9be6 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51082024-12-14T04:49:52.861ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2560 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 47a388f0-8abe-4c2e-958a-08187f6c9be6 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51092024-12-14T04:49:53.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = faef430f-1e9f-43fe-9ac3-8e03806c61ef roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51102024-12-14T04:49:53.881ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = faef430f-1e9f-43fe-9ac3-8e03806c61ef resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51112024-12-14T04:49:53.884ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2991 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = faef430f-1e9f-43fe-9ac3-8e03806c61ef response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51122024-12-14T04:49:54.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ee244994-4ef4-4e7f-a6ca-969fb85f1525 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51132024-12-14T04:49:54.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ee244994-4ef4-4e7f-a6ca-969fb85f1525 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51142024-12-14T04:49:54.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1729 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ee244994-4ef4-4e7f-a6ca-969fb85f1525 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51152024-12-14T04:49:55.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a45ce6e3-0f8c-4ff0-9ca5-f7359d780a49 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51162024-12-14T04:49:55.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a45ce6e3-0f8c-4ff0-9ca5-f7359d780a49 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51172024-12-14T04:49:55.928ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2649 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a45ce6e3-0f8c-4ff0-9ca5-f7359d780a49 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51182024-12-14T04:49:56.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
51192024-12-14T04:49:56.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51202024-12-14T04:49:56.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51212024-12-14T04:49:56.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
51222024-12-14T04:49:56.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51232024-12-14T04:49:56.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51242024-12-14T04:49:56.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51252024-12-14T04:49:56.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51262024-12-14T04:49:56.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51272024-12-14T04:49:56.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.314742ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51282024-12-14T04:49:56.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51292024-12-14T04:49:56.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51302024-12-14T04:49:56.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51312024-12-14T04:49:56.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51322024-12-14T04:49:56.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51332024-12-14T04:49:56.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51342024-12-14T04:49:56.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
51352024-12-14T04:49:56.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51362024-12-14T04:49:56.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51372024-12-14T04:49:56.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
51382024-12-14T04:49:56.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.056256ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
51392024-12-14T04:49:56.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 605b84dd-b2bf-49b7-b999-be05ec15eaa1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51402024-12-14T04:49:56.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 605b84dd-b2bf-49b7-b999-be05ec15eaa1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51412024-12-14T04:49:56.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1927 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 605b84dd-b2bf-49b7-b999-be05ec15eaa1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51422024-12-14T04:49:57.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 487fce2d-2dd0-4b05-943d-044eb2d24486 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51432024-12-14T04:49:57.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 487fce2d-2dd0-4b05-943d-044eb2d24486 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51442024-12-14T04:49:57.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2415 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 487fce2d-2dd0-4b05-943d-044eb2d24486 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51452024-12-14T04:49:58.993ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8423445-8651-46e7-a9bf-fc0a4356a409 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51462024-12-14T04:49:58.993ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8423445-8651-46e7-a9bf-fc0a4356a409 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51472024-12-14T04:49:58.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1878 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d8423445-8651-46e7-a9bf-fc0a4356a409 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51482024-12-14T04:50:00.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9fa8545c-210f-4464-b8f7-e971a44e3b9e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51492024-12-14T04:50:00.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9fa8545c-210f-4464-b8f7-e971a44e3b9e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51502024-12-14T04:50:00.017ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2126 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9fa8545c-210f-4464-b8f7-e971a44e3b9e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51512024-12-14T04:50:01.037ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bcd339cc-b2a9-4aa2-ac22-7a80a551be99 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51522024-12-14T04:50:01.037ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bcd339cc-b2a9-4aa2-ac22-7a80a551be99 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51532024-12-14T04:50:01.039ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2488 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bcd339cc-b2a9-4aa2-ac22-7a80a551be99 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51542024-12-14T04:50:02.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = acf99850-2fcb-4e10-a179-ee0c4829d2bc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51552024-12-14T04:50:02.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = acf99850-2fcb-4e10-a179-ee0c4829d2bc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51562024-12-14T04:50:02.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2668 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = acf99850-2fcb-4e10-a179-ee0c4829d2bc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51572024-12-14T04:50:02.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1369 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 983807c8-6c5f-4897-808d-750da0170c51 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
51582024-12-14T04:50:03.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ed61bd4e-e38c-475c-85fc-4420fc23cd3e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51592024-12-14T04:50:03.083ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ed61bd4e-e38c-475c-85fc-4420fc23cd3e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51602024-12-14T04:50:03.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2393 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ed61bd4e-e38c-475c-85fc-4420fc23cd3e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51612024-12-14T04:50:04.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 172f3650-bf58-4d23-a094-9c71405da737 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51622024-12-14T04:50:04.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 172f3650-bf58-4d23-a094-9c71405da737 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51632024-12-14T04:50:04.107ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1980 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 172f3650-bf58-4d23-a094-9c71405da737 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51642024-12-14T04:50:05.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 50f5b5e5-1239-428b-a414-5afd949543e1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51652024-12-14T04:50:05.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 50f5b5e5-1239-428b-a414-5afd949543e1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51662024-12-14T04:50:05.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2220 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 50f5b5e5-1239-428b-a414-5afd949543e1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51672024-12-14T04:50:06.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e777c0a6-883a-4cb6-b242-a56b58fd2188 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51682024-12-14T04:50:06.150ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e777c0a6-883a-4cb6-b242-a56b58fd2188 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51692024-12-14T04:50:06.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2587 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e777c0a6-883a-4cb6-b242-a56b58fd2188 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
51702024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
51712024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
51722024-12-14T04:50:06.882ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 04:40:06.882657257 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
51732024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
51742024-12-14T04:50:06.882ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
51752024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
51762024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51772024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
51782024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 17 reason = Timeout
51792024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 17 reason = Timeout
51802024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51812024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 17 reason = Timeout
51822024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
51832024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51842024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
51852024-12-14T04:50:06.882ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832
51862024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.761\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832"}
51872024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
51882024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
51892024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
51902024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
51912024-12-14T04:50:06.882ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
51922024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
51932024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
51942024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
51952024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51962024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
51972024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
51982024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
51992024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52002024-12-14T04:50:06.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
52012024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
52022024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
52032024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
52042024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
52052024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52062024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
52072024-12-14T04:50:06.883ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
52082024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52092024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
52102024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
52112024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
52122024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52132024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52142024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Timeout
52152024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
52162024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
52172024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52182024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
52192024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 17 reason = Timeout
52202024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
52212024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52222024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52232024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
52242024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
52252024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52262024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
52272024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52282024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
52292024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
52302024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
52312024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
52322024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52332024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52342024-12-14T04:50:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6ba1572d-df24-4bd2-be03-9a99edc867d8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b0cf818e-eb68-4273-b879-3df82709e7d6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1580, tv_nsec: 979564396 } })
52352024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52362024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
52372024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
52382024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52392024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
52402024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
52412024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52422024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 17 reason = Timeout
52432024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52442024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52452024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
52462024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
52472024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52482024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
52492024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52502024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
52512024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
52522024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52532024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52542024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52552024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52562024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
52572024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
52582024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52592024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52602024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52612024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52622024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52632024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
52642024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
52652024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.063408ms iteration = 17 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
52662024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
52672024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52682024-12-14T04:50:06.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2224041a-a9fc-46d2-b9ed-ef98dc01f696.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("229c9d57-e318-46f6-8df9-9f602a126f12.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5f1ff79-58eb-4277-8805-3948cd3d20e1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1580, tv_nsec: 980173932 } })
52692024-12-14T04:50:06.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52702024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52712024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52722024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52732024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52742024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
52752024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 941.315\u{b5}s iteration = 17 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
52762024-12-14T04:50:06.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52772024-12-14T04:50:06.885ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 04:40:06.882657257 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
52782024-12-14T04:50:06.885ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.07103ms iteration = 16 status = {"expiration":"2024-12-14T04:40:06.882657257Z","pruned":[]}
52792024-12-14T04:50:06.886ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52802024-12-14T04:50:06.886ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1580, tv_nsec: 982903453 } })
52812024-12-14T04:50:06.886ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.84246ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
52822024-12-14T04:50:06.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52832024-12-14T04:50:06.886ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
52842024-12-14T04:50:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52852024-12-14T04:50:06.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52862024-12-14T04:50:06.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52872024-12-14T04:50:06.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
52882024-12-14T04:50:06.888ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52892024-12-14T04:50:06.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52902024-12-14T04:50:06.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
52912024-12-14T04:50:06.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52922024-12-14T04:50:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52932024-12-14T04:50:06.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
52942024-12-14T04:50:06.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52952024-12-14T04:50:06.890ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
52962024-12-14T04:50:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52972024-12-14T04:50:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
52982024-12-14T04:50:06.891ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
52992024-12-14T04:50:06.891ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
53002024-12-14T04:50:06.891ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53012024-12-14T04:50:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53022024-12-14T04:50:06.892ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53032024-12-14T04:50:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53042024-12-14T04:50:06.893ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.562775ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
53052024-12-14T04:50:06.894ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53062024-12-14T04:50:06.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53072024-12-14T04:50:06.895ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1580, tv_nsec: 991823095 } })
53082024-12-14T04:50:06.895ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
53092024-12-14T04:50:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53102024-12-14T04:50:06.896ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53112024-12-14T04:50:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53122024-12-14T04:50:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
53132024-12-14T04:50:06.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53142024-12-14T04:50:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53152024-12-14T04:50:06.899ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53162024-12-14T04:50:06.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53172024-12-14T04:50:06.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53182024-12-14T04:50:06.903ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53192024-12-14T04:50:06.903ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53202024-12-14T04:50:06.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
53212024-12-14T04:50:06.905ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
53222024-12-14T04:50:06.905ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53232024-12-14T04:50:06.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
53242024-12-14T04:50:06.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53252024-12-14T04:50:06.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53262024-12-14T04:50:06.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53272024-12-14T04:50:06.912ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53282024-12-14T04:50:06.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53292024-12-14T04:50:06.914ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53302024-12-14T04:50:06.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53312024-12-14T04:50:06.915ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
53322024-12-14T04:50:06.915ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
53332024-12-14T04:50:06.915ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 32.61191ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
53342024-12-14T04:50:06.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53352024-12-14T04:50:06.915ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53362024-12-14T04:50:06.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53372024-12-14T04:50:06.916ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
53382024-12-14T04:50:06.916ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53392024-12-14T04:50:06.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
53402024-12-14T04:50:06.917ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53412024-12-14T04:50:06.918ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53422024-12-14T04:50:06.918ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53432024-12-14T04:50:06.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53442024-12-14T04:50:06.920ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53452024-12-14T04:50:06.920ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
53462024-12-14T04:50:06.920ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53472024-12-14T04:50:06.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53482024-12-14T04:50:06.922ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53492024-12-14T04:50:06.922ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53502024-12-14T04:50:06.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53512024-12-14T04:50:06.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 39.385677ms iteration = 31 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
53522024-12-14T04:50:06.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53532024-12-14T04:50:06.924ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 41.85276ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
53542024-12-14T04:50:06.924ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53552024-12-14T04:50:06.924ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53562024-12-14T04:50:06.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53572024-12-14T04:50:06.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53582024-12-14T04:50:06.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53592024-12-14T04:50:06.927ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
53602024-12-14T04:50:06.927ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 43.694879ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
53612024-12-14T04:50:06.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
53622024-12-14T04:50:06.927ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53632024-12-14T04:50:06.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53642024-12-14T04:50:06.929ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
53652024-12-14T04:50:06.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
53662024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53672024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53682024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 57.493558ms iteration = 31 status = {"errors":[],"records_set_to_done":[]}
53692024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53702024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53712024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53722024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53732024-12-14T04:50:06.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
53742024-12-14T04:50:06.941ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
53752024-12-14T04:50:06.941ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
53762024-12-14T04:50:06.941ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 58.091673ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
53772024-12-14T04:50:06.941ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53782024-12-14T04:50:06.942ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53792024-12-14T04:50:06.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53802024-12-14T04:50:06.942ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 59.724847ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
53812024-12-14T04:50:06.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53822024-12-14T04:50:06.943ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53832024-12-14T04:50:06.943ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
53842024-12-14T04:50:06.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
53852024-12-14T04:50:06.944ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53862024-12-14T04:50:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53872024-12-14T04:50:06.945ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53882024-12-14T04:50:06.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53892024-12-14T04:50:06.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53902024-12-14T04:50:06.946ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53912024-12-14T04:50:06.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53922024-12-14T04:50:06.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
53932024-12-14T04:50:06.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53942024-12-14T04:50:06.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53952024-12-14T04:50:06.948ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53962024-12-14T04:50:06.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
53972024-12-14T04:50:06.949ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53982024-12-14T04:50:06.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53992024-12-14T04:50:06.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
54002024-12-14T04:50:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
54012024-12-14T04:50:06.951ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54022024-12-14T04:50:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54032024-12-14T04:50:06.954ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54042024-12-14T04:50:06.956ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
54052024-12-14T04:50:06.956ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54062024-12-14T04:50:06.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54072024-12-14T04:50:06.957ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54082024-12-14T04:50:06.959ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54092024-12-14T04:50:06.959ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1581, tv_nsec: 56129581 } })
54102024-12-14T04:50:06.960ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
54112024-12-14T04:50:06.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
54122024-12-14T04:50:06.960ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54132024-12-14T04:50:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54142024-12-14T04:50:06.961ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54152024-12-14T04:50:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
54162024-12-14T04:50:06.963ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54172024-12-14T04:50:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54182024-12-14T04:50:06.963ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54192024-12-14T04:50:06.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
54202024-12-14T04:50:06.963ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
54212024-12-14T04:50:06.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
54222024-12-14T04:50:06.965ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54232024-12-14T04:50:06.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54242024-12-14T04:50:06.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54252024-12-14T04:50:06.966ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 83.550097ms iteration = 7 status = {"nsuccess":5}
54262024-12-14T04:50:06.971ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
54272024-12-14T04:50:06.971ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
54282024-12-14T04:50:06.971ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
54292024-12-14T04:50:06.971ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54302024-12-14T04:50:06.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54312024-12-14T04:50:06.972ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54322024-12-14T04:50:06.974ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54332024-12-14T04:50:06.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54342024-12-14T04:50:06.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54352024-12-14T04:50:06.976ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 576b2424-f667-449e-9201-f877adde32d0 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
54362024-12-14T04:50:06.976ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 576b2424-f667-449e-9201-f877adde32d0 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
54372024-12-14T04:50:06.976ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
54382024-12-14T04:50:06.976ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.561388ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
54392024-12-14T04:50:06.979ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54402024-12-14T04:50:06.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54412024-12-14T04:50:06.984ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54422024-12-14T04:50:06.984ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
54432024-12-14T04:50:06.984ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 101.369511ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
54442024-12-14T04:50:06.989ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
54452024-12-14T04:50:06.990ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 107.131451ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54462024-12-14T04:50:06.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
54472024-12-14T04:50:06.992ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 109.699178ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54482024-12-14T04:50:06.994ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 111.923437ms iteration = 35 status = {"physical_disks_added":0}
54492024-12-14T04:50:06.996ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
54502024-12-14T04:50:07.004ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
54512024-12-14T04:50:07.004ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
54522024-12-14T04:50:07.004ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
54532024-12-14T04:50:07.004ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54542024-12-14T04:50:07.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54552024-12-14T04:50:07.020ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
54562024-12-14T04:50:07.033ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
54572024-12-14T04:50:07.033ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
54582024-12-14T04:50:07.034ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4035914-6c49-4c9b-85bd-267d6df80592", "content-length": "26", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} })
54592024-12-14T04:50:07.034ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
54602024-12-14T04:50:07.036ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
54612024-12-14T04:50:07.040ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
54622024-12-14T04:50:07.040ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
54632024-12-14T04:50:07.040ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f66db55d-6408-4619-b088-b5c49c32e394", "content-length": "26", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} })
54642024-12-14T04:50:07.040ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
54652024-12-14T04:50:07.042ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
54662024-12-14T04:50:07.120ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
54672024-12-14T04:50:07.120ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
54682024-12-14T04:50:07.121ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2bcff9a-b353-469e-9ded-e5af90eadfc6", "content-length": "26", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} })
54692024-12-14T04:50:07.121ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
54702024-12-14T04:50:07.122ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
54712024-12-14T04:50:07.123ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
54722024-12-14T04:50:07.123ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d30a747-062f-40de-a809-d9f8ac1226fa", "content-length": "691", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
54732024-12-14T04:50:07.123ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
54742024-12-14T04:50:07.124ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
54752024-12-14T04:50:07.124ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
54762024-12-14T04:50:07.125ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
54772024-12-14T04:50:07.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
54782024-12-14T04:50:07.125ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 243.201617ms iteration = 32 status = {}
54792024-12-14T04:50:07.126ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54802024-12-14T04:50:07.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54812024-12-14T04:50:07.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54822024-12-14T04:50:07.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152858 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 576b2424-f667-449e-9201-f877adde32d0 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
54832024-12-14T04:50:07.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54842024-12-14T04:50:07.129ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54852024-12-14T04:50:07.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54862024-12-14T04:50:07.131ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54872024-12-14T04:50:07.131ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
54882024-12-14T04:50:07.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
54892024-12-14T04:50:07.132ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54902024-12-14T04:50:07.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54912024-12-14T04:50:07.134ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54922024-12-14T04:50:07.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54932024-12-14T04:50:07.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
54942024-12-14T04:50:07.134ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54952024-12-14T04:50:07.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54962024-12-14T04:50:07.136ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 253.729652ms iteration = 31 status = 0
54972024-12-14T04:50:07.156ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54982024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
54992024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55002024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55012024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
55022024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 0 version = 2
55032024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55042024-12-14T04:50:07.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55052024-12-14T04:50:07.162ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
55062024-12-14T04:50:07.162ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
55072024-12-14T04:50:07.162ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55082024-12-14T04:50:07.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55092024-12-14T04:50:07.174ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6bfcd190-d3a4-4bbb-abd0-000f128b03cd roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
55102024-12-14T04:50:07.174ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6bfcd190-d3a4-4bbb-abd0-000f128b03cd resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
55112024-12-14T04:50:07.176ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1943 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 6bfcd190-d3a4-4bbb-abd0-000f128b03cd response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
55122024-12-14T04:50:07.176ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55132024-12-14T04:50:07.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
55142024-12-14T04:50:07.180ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55152024-12-14T04:50:07.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55162024-12-14T04:50:07.182ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55172024-12-14T04:50:07.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55182024-12-14T04:50:07.184ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55192024-12-14T04:50:07.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55202024-12-14T04:50:07.186ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55212024-12-14T04:50:07.186ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55222024-12-14T04:50:07.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55232024-12-14T04:50:07.188ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55242024-12-14T04:50:07.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
55252024-12-14T04:50:07.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55262024-12-14T04:50:07.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55272024-12-14T04:50:07.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55282024-12-14T04:50:07.192ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
55292024-12-14T04:50:07.200ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T04:35:03.711237Z, time_modified: 2024-12-14T04:35:03.711237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 79c3fe08-86fc-4863-afcc-4cee91996db1, db_silo: Silo { identity: SiloIdentity { id: 79c3fe08-86fc-4863-afcc-4cee91996db1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T04:35:19.650112Z, time_modified: 2024-12-14T04:35:19.650112Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1" }] }) }
55302024-12-14T04:50:07.201ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 318.292844ms iteration = 17 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]}},"default_endpoint":{"silo_id":"79c3fe08-86fc-4863-afcc-4cee91996db1","tls_certs":["e0f0db6f1c5d8464f803ad1f614b24b5d2e29dcc4238b18e2c346176f5ffb5d1"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
55312024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55322024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55332024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 162c8782-35cf-4bcd-a6cc-cc340fb51981 dns_zone_name = oxide.test found_so_far = 1 version = 2
55342024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
55352024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55362024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 04:35:19.721992 UTC dns_group = external generation = 2
55372024-12-14T04:50:07.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 319.885246ms iteration = 17 status = {"generation":2}
55382024-12-14T04:50:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
55392024-12-14T04:50:07.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55402024-12-14T04:50:07.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55412024-12-14T04:50:07.206ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55422024-12-14T04:50:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55432024-12-14T04:50:07.207ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55442024-12-14T04:50:07.207ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 38d44253-0d31-4f00-9b07-0a3a137d3d5b dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
55452024-12-14T04:50:07.207ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
55462024-12-14T04:50:07.207ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 04:35:18.025163 UTC dns_group = internal generation = 1
55472024-12-14T04:50:07.207ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 324.642129ms iteration = 17 status = {"generation":1}
55482024-12-14T04:50:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
55492024-12-14T04:50:07.210ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55502024-12-14T04:50:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55512024-12-14T04:50:07.213ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55522024-12-14T04:50:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55532024-12-14T04:50:07.215ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55542024-12-14T04:50:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55552024-12-14T04:50:07.217ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55562024-12-14T04:50:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55572024-12-14T04:50:07.219ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55582024-12-14T04:50:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55592024-12-14T04:50:07.221ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55602024-12-14T04:50:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55612024-12-14T04:50:07.222ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55622024-12-14T04:50:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55632024-12-14T04:50:07.224ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55642024-12-14T04:50:07.227ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
55652024-12-14T04:50:07.227ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55662024-12-14T04:50:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55672024-12-14T04:50:07.228ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55682024-12-14T04:50:07.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55692024-12-14T04:50:07.230ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55702024-12-14T04:50:07.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55712024-12-14T04:50:07.231ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55722024-12-14T04:50:07.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
55732024-12-14T04:50:07.234ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55742024-12-14T04:50:07.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55752024-12-14T04:50:07.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55762024-12-14T04:50:07.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55772024-12-14T04:50:07.240ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55782024-12-14T04:50:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55792024-12-14T04:50:07.242ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55802024-12-14T04:50:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55812024-12-14T04:50:07.242ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 359.58726ms iteration = 31 status = {}
55822024-12-14T04:50:07.244ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55832024-12-14T04:50:07.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55842024-12-14T04:50:07.245ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
55852024-12-14T04:50:07.245ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 361.649493ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
55862024-12-14T04:50:07.246ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55872024-12-14T04:50:07.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55882024-12-14T04:50:07.247ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55892024-12-14T04:50:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55902024-12-14T04:50:07.248ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55912024-12-14T04:50:07.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55922024-12-14T04:50:07.249ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55932024-12-14T04:50:07.252ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
55942024-12-14T04:50:07.252ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7d01ef95-9724-4a0c-a3ce-0c1447f6acfe", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55952024-12-14T04:50:07.252ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55962024-12-14T04:50:07.253ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
55972024-12-14T04:50:07.253ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55982024-12-14T04:50:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55992024-12-14T04:50:07.254ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56002024-12-14T04:50:07.254ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "75957570-130a-413e-8bee-874dfa7347e8", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56012024-12-14T04:50:07.254ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56022024-12-14T04:50:07.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56032024-12-14T04:50:07.255ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56042024-12-14T04:50:07.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56052024-12-14T04:50:07.257ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56062024-12-14T04:50:07.257ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 374.419196ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
56072024-12-14T04:50:07.257ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56082024-12-14T04:50:07.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
56092024-12-14T04:50:07.261ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56102024-12-14T04:50:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56112024-12-14T04:50:07.266ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0d2b4f8f-354d-4398-8d3a-37472bac15e3, kind: Service { id: b0cf818e-eb68-4273-b879-3df82709e7d6 }, name: Name("external-dns-b0cf818e-eb68-4273-b879-3df82709e7d6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 149])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 749b346b-1343-4e3a-a01e-e19ce433ac08, kind: Service { id: 6ba1572d-df24-4bd2-be03-9a99edc867d8 }, name: Name("external-dns-6ba1572d-df24-4bd2-be03-9a99edc867d8"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 884c1dd6-beb8-4af4-a5a6-31995ecbb183, kind: Service { id: bb43c8e4-f24f-424b-aa30-a7b5d215708e }, name: Name("nexus-bb43c8e4-f24f-424b-aa30-a7b5d215708e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 185])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 985a6f50-eb4b-450e-8994-8e677143cd87, kind: Service { id: e36fd091-efce-4ea5-9e73-a9bc4cb7a549 }, name: Name("nexus-e36fd091-efce-4ea5-9e73-a9bc4cb7a549"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 234])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f0e8ac24-f32d-4488-9c3c-9af00b9bda6f, kind: Service { id: a2394d1d-754d-4ed4-8218-d02ec0d792df }, name: Name("nexus-a2394d1d-754d-4ed4-8218-d02ec0d792df"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 95])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
56122024-12-14T04:50:07.266ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56132024-12-14T04:50:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56142024-12-14T04:50:07.268ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56152024-12-14T04:50:07.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
56162024-12-14T04:50:07.269ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56172024-12-14T04:50:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56182024-12-14T04:50:07.271ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
56192024-12-14T04:50:07.271ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 0d2b4f8f-354d-4398-8d3a-37472bac15e3, kind: Service { id: b0cf818e-eb68-4273-b879-3df82709e7d6 }, name: Name("external-dns-b0cf818e-eb68-4273-b879-3df82709e7d6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 149])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 749b346b-1343-4e3a-a01e-e19ce433ac08, kind: Service { id: 6ba1572d-df24-4bd2-be03-9a99edc867d8 }, name: Name("external-dns-6ba1572d-df24-4bd2-be03-9a99edc867d8"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 884c1dd6-beb8-4af4-a5a6-31995ecbb183, kind: Service { id: bb43c8e4-f24f-424b-aa30-a7b5d215708e }, name: Name("nexus-bb43c8e4-f24f-424b-aa30-a7b5d215708e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 185])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 985a6f50-eb4b-450e-8994-8e677143cd87, kind: Service { id: e36fd091-efce-4ea5-9e73-a9bc4cb7a549 }, name: Name("nexus-e36fd091-efce-4ea5-9e73-a9bc4cb7a549"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 234])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f0e8ac24-f32d-4488-9c3c-9af00b9bda6f, kind: Service { id: a2394d1d-754d-4ed4-8218-d02ec0d792df }, name: Name("nexus-a2394d1d-754d-4ed4-8218-d02ec0d792df"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 95])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
56202024-12-14T04:50:07.271ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
56212024-12-14T04:50:07.278ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
56222024-12-14T04:50:07.279ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d292f7a0-bf56-4c87-a2c7-00f9f3bb47ce", "content-length": "2", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56232024-12-14T04:50:07.284ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
56242024-12-14T04:50:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
56252024-12-14T04:50:07.285ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 403.099223ms iteration = 31 status = null
56262024-12-14T04:50:07.307ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
56272024-12-14T04:50:07.307ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56282024-12-14T04:50:07.307ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "adfc8294-e678-400b-a07c-fa2d760d78ef", "content-length": "372", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56292024-12-14T04:50:07.307ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56302024-12-14T04:50:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56312024-12-14T04:50:07.310ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
56322024-12-14T04:50:07.311ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bade414e-7c1d-448e-9e95-8340bb0ebf4f", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56332024-12-14T04:50:07.311ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56342024-12-14T04:50:07.313ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b227f547-6cc4-4ea5-b3fd-3893efdd0892", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56352024-12-14T04:50:07.313ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56362024-12-14T04:50:07.315ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "042bad40-9780-4524-84f8-efb4a001ee8c", "date": "Sat, 14 Dec 2024 04:50:06 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56372024-12-14T04:50:07.315ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56382024-12-14T04:50:07.317ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
56392024-12-14T04:50:07.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 434.129202ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
56402024-12-14T04:50:07.347ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
56412024-12-14T04:50:07.347ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
56422024-12-14T04:50:07.347ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5588720d-f47a-413d-8951-2958853836c3", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56432024-12-14T04:50:07.347ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56442024-12-14T04:50:07.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56452024-12-14T04:50:07.350ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56462024-12-14T04:50:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56472024-12-14T04:50:07.353ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56482024-12-14T04:50:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56492024-12-14T04:50:07.357ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
56502024-12-14T04:50:07.357ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56512024-12-14T04:50:07.357ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "de449f38-3b55-4b1f-af53-89895e5aa6c5", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56522024-12-14T04:50:07.359ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 476.6661ms iteration = 33 status = {}
56532024-12-14T04:50:07.362ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56542024-12-14T04:50:07.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56552024-12-14T04:50:07.363ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
56562024-12-14T04:50:07.363ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56572024-12-14T04:50:07.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56582024-12-14T04:50:07.373ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
56592024-12-14T04:50:07.373ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4572ed8-9738-4cec-9bc3-8187775bea48", "content-length": "134", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56602024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
56612024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
56622024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
56632024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
56642024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
56652024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
56662024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
56672024-12-14T04:50:07.373ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
56682024-12-14T04:50:07.373ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, time_created: 2024-12-14T04:35:18.598624Z, time_modified: 2024-12-14T04:35:18.598624Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
56692024-12-14T04:50:07.373ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
56702024-12-14T04:50:07.373ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56712024-12-14T04:50:07.374ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
56722024-12-14T04:50:07.374ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
56732024-12-14T04:50:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56742024-12-14T04:50:07.374ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
56752024-12-14T04:50:07.374ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56762024-12-14T04:50:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56772024-12-14T04:50:07.375ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56782024-12-14T04:50:07.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56792024-12-14T04:50:07.375ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
56802024-12-14T04:50:07.376ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "01c9f2c1-4221-4832-89f6-6051ac60d137", "content-length": "2", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56812024-12-14T04:50:07.376ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
56822024-12-14T04:50:07.376ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bdbcebb-4e3a-4b3e-bf5b-448c11c19683", "content-length": "52", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56832024-12-14T04:50:07.376ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
56842024-12-14T04:50:07.376ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
56852024-12-14T04:50:07.377ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56862024-12-14T04:50:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
56872024-12-14T04:50:07.411ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
56882024-12-14T04:50:07.473ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "877db373-5806-4197-b930-e25a902a32ea", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56892024-12-14T04:50:07.473ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
56902024-12-14T04:50:07.473ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
56912024-12-14T04:50:07.473ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "343510d0-14e2-4255-9c72-469527458ff0", "content-length": "682", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56922024-12-14T04:50:07.473ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56932024-12-14T04:50:07.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56942024-12-14T04:50:07.474ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "a63b8f4e-0adc-41f2-80f0-d644ea77131c", "date": "Sat, 14 Dec 2024 04:50:07 GMT"} })
56952024-12-14T04:50:07.475ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
56962024-12-14T04:50:07.475ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 592.768136ms
56972024-12-14T04:50:07.475ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 592.778456ms iteration = 4 status = {}
56982024-12-14T04:50:07.477ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56992024-12-14T04:50:07.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57002024-12-14T04:50:07.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57012024-12-14T04:50:07.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57022024-12-14T04:50:07.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57032024-12-14T04:50:07.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57042024-12-14T04:50:07.491ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 608.766282ms iteration = 31 status = {}
57052024-12-14T04:50:08.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c116e5c4-2595-4c1f-bbf9-9c01e098e21e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57062024-12-14T04:50:08.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c116e5c4-2595-4c1f-bbf9-9c01e098e21e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57072024-12-14T04:50:08.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2524 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c116e5c4-2595-4c1f-bbf9-9c01e098e21e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57082024-12-14T04:50:09.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7bb47158-7b4e-4375-9ebb-50daceac7e54 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57092024-12-14T04:50:09.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7bb47158-7b4e-4375-9ebb-50daceac7e54 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57102024-12-14T04:50:09.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2065 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7bb47158-7b4e-4375-9ebb-50daceac7e54 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57112024-12-14T04:50:10.244ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5868a3bd-6c1e-4d14-86ed-a332fdf5daf4 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57122024-12-14T04:50:10.245ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5868a3bd-6c1e-4d14-86ed-a332fdf5daf4 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57132024-12-14T04:50:10.247ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3169 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5868a3bd-6c1e-4d14-86ed-a332fdf5daf4 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57142024-12-14T04:50:11.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 76143ef8-8d86-4866-ac4d-efcfbe37a8a6 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57152024-12-14T04:50:11.268ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 76143ef8-8d86-4866-ac4d-efcfbe37a8a6 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57162024-12-14T04:50:11.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2162 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 76143ef8-8d86-4866-ac4d-efcfbe37a8a6 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57172024-12-14T04:50:12.290ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 28289879-6cf9-48c4-9572-6dc84169872c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57182024-12-14T04:50:12.290ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 28289879-6cf9-48c4-9572-6dc84169872c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57192024-12-14T04:50:12.293ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3050 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 28289879-6cf9-48c4-9572-6dc84169872c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57202024-12-14T04:50:12.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1487 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 026019fe-8df2-4c8c-827b-4828260ac471 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
57212024-12-14T04:50:13.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 17bbdb95-6d5f-4319-9495-6c651b5d46fc roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57222024-12-14T04:50:13.314ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 17bbdb95-6d5f-4319-9495-6c651b5d46fc resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57232024-12-14T04:50:13.316ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2268 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 17bbdb95-6d5f-4319-9495-6c651b5d46fc response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57242024-12-14T04:50:14.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3eb228ea-e85f-4f7b-84a9-0c64d6fdd49e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57252024-12-14T04:50:14.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3eb228ea-e85f-4f7b-84a9-0c64d6fdd49e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57262024-12-14T04:50:14.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2057 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3eb228ea-e85f-4f7b-84a9-0c64d6fdd49e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57272024-12-14T04:50:15.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c75e5056-3d14-4f20-b1ea-98081dd573aa roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57282024-12-14T04:50:15.358ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c75e5056-3d14-4f20-b1ea-98081dd573aa resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57292024-12-14T04:50:15.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2126 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = c75e5056-3d14-4f20-b1ea-98081dd573aa response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57302024-12-14T04:50:16.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a70eb5a8-392b-4761-af75-31599f74b152 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57312024-12-14T04:50:16.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a70eb5a8-392b-4761-af75-31599f74b152 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57322024-12-14T04:50:16.381ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1888 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = a70eb5a8-392b-4761-af75-31599f74b152 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57332024-12-14T04:50:16.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 92 reason = Timeout
57342024-12-14T04:50:16.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57352024-12-14T04:50:16.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57362024-12-14T04:50:16.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 92 reason = Timeout
57372024-12-14T04:50:16.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57382024-12-14T04:50:16.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57392024-12-14T04:50:16.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57402024-12-14T04:50:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57412024-12-14T04:50:16.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57422024-12-14T04:50:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57432024-12-14T04:50:16.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57442024-12-14T04:50:16.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57452024-12-14T04:50:16.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57462024-12-14T04:50:16.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
57472024-12-14T04:50:16.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57482024-12-14T04:50:16.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57492024-12-14T04:50:16.903ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
57502024-12-14T04:50:16.903ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.459897ms iteration = 92 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
57512024-12-14T04:50:16.913ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57522024-12-14T04:50:16.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57532024-12-14T04:50:16.941ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.928489ms iteration = 92 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
57542024-12-14T04:50:17.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 86364e23-6449-4cbc-80f2-537fb6784433 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57552024-12-14T04:50:17.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 86364e23-6449-4cbc-80f2-537fb6784433 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57562024-12-14T04:50:17.402ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2618 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 86364e23-6449-4cbc-80f2-537fb6784433 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57572024-12-14T04:50:17.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c8010cc3-0241-4b36-93d7-1d855bc505ed roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
57582024-12-14T04:50:17.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c8010cc3-0241-4b36-93d7-1d855bc505ed resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
57592024-12-14T04:50:17.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2993 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = c8010cc3-0241-4b36-93d7-1d855bc505ed response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
57602024-12-14T04:50:17.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = cb02f9da-1d2c-49ba-9a4c-2060c7d0370d roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
57612024-12-14T04:50:17.774ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = cb02f9da-1d2c-49ba-9a4c-2060c7d0370d resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
57622024-12-14T04:50:17.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2629 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = cb02f9da-1d2c-49ba-9a4c-2060c7d0370d response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
57632024-12-14T04:50:18.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5fb1781b-82eb-4fc3-86a7-5179bb896533 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57642024-12-14T04:50:18.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5fb1781b-82eb-4fc3-86a7-5179bb896533 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57652024-12-14T04:50:18.425ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2371 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 5fb1781b-82eb-4fc3-86a7-5179bb896533 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57662024-12-14T04:50:19.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fa2f8d6e-7c7e-4661-84ee-062b4ccdefb2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57672024-12-14T04:50:19.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fa2f8d6e-7c7e-4661-84ee-062b4ccdefb2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57682024-12-14T04:50:19.448ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2779 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = fa2f8d6e-7c7e-4661-84ee-062b4ccdefb2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57692024-12-14T04:50:20.344ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:39977
57702024-12-14T04:50:20.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:39977 req_id = a1ef8d6b-d812-422c-af46-4846c66f3a0d roles = RoleSet { roles: {} } uri = /metrics/producers
57712024-12-14T04:50:20.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:39977 req_id = a1ef8d6b-d812-422c-af46-4846c66f3a0d resource = Database result = Ok(()) uri = /metrics/producers
57722024-12-14T04:50:20.443ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 5f4909e4-dcea-4db6-99c3-a64609b9f774
57732024-12-14T04:50:20.443ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99770 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:39977 req_id = a1ef8d6b-d812-422c-af46-4846c66f3a0d response_code = 201 uri = /metrics/producers
57742024-12-14T04:50:20.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56a2e37f-c31c-4dc1-bf91-5c0d47f18f01 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57752024-12-14T04:50:20.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56a2e37f-c31c-4dc1-bf91-5c0d47f18f01 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57762024-12-14T04:50:20.470ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2293 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 56a2e37f-c31c-4dc1-bf91-5c0d47f18f01 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57772024-12-14T04:50:21.489ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1e69d508-50d2-451b-929b-696197ccc6cb roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57782024-12-14T04:50:21.489ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1e69d508-50d2-451b-929b-696197ccc6cb resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57792024-12-14T04:50:21.491ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1e69d508-50d2-451b-929b-696197ccc6cb response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57802024-12-14T04:50:22.452ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
57812024-12-14T04:50:22.452ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
57822024-12-14T04:50:22.483ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
57832024-12-14T04:50:22.483ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:50460
57842024-12-14T04:50:22.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50460 req_id = 38299d31-9f2a-4509-b39b-2ea1fbb4def5 roles = RoleSet { roles: {} } uri = /metrics/producers
57852024-12-14T04:50:22.484ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50460 req_id = 38299d31-9f2a-4509-b39b-2ea1fbb4def5 resource = Database result = Ok(()) uri = /metrics/producers
57862024-12-14T04:50:22.511ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 474dbab9-93d0-4bea-9ede-12b9ea4e90c1 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57872024-12-14T04:50:22.511ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 474dbab9-93d0-4bea-9ede-12b9ea4e90c1 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57882024-12-14T04:50:22.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2309 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 474dbab9-93d0-4bea-9ede-12b9ea4e90c1 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57892024-12-14T04:50:22.543ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = a2394d1d-754d-4ed4-8218-d02ec0d792df
57902024-12-14T04:50:22.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59261 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50460 req_id = 38299d31-9f2a-4509-b39b-2ea1fbb4def5 response_code = 201 uri = /metrics/producers
57912024-12-14T04:50:22.543ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "38299d31-9f2a-4509-b39b-2ea1fbb4def5", "content-length": "41", "date": "Sat, 14 Dec 2024 04:50:22 GMT"} })
57922024-12-14T04:50:22.544ZDEBGnexus (producer-registration-task): registered with nexus successfully
57932024-12-14T04:50:22.544ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
57942024-12-14T04:50:22.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1513 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 321a7de2-6f0e-4b42-bc0d-6412eea0bb7e response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
57952024-12-14T04:50:23.532ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1ccce008-cad4-4ebf-b698-4b1c4ca59ce2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57962024-12-14T04:50:23.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1ccce008-cad4-4ebf-b698-4b1c4ca59ce2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57972024-12-14T04:50:23.535ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2713 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 1ccce008-cad4-4ebf-b698-4b1c4ca59ce2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
57982024-12-14T04:50:24.293ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:38277
57992024-12-14T04:50:24.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38277 req_id = 7e021145-428c-4a64-b3c3-1adcc3fc2824 roles = RoleSet { roles: {} } uri = /metrics/producers
58002024-12-14T04:50:24.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38277 req_id = 7e021145-428c-4a64-b3c3-1adcc3fc2824 resource = Database result = Ok(()) uri = /metrics/producers
58012024-12-14T04:50:24.333ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = b1576088-d04d-48cf-a285-75e4628d77d5
58022024-12-14T04:50:24.333ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39872 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:38277 req_id = 7e021145-428c-4a64-b3c3-1adcc3fc2824 response_code = 201 uri = /metrics/producers
58032024-12-14T04:50:24.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7bc41e79-eb7a-4f2b-9e29-42e01e358fd3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58042024-12-14T04:50:24.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7bc41e79-eb7a-4f2b-9e29-42e01e358fd3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58052024-12-14T04:50:24.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2255 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7bc41e79-eb7a-4f2b-9e29-42e01e358fd3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58062024-12-14T04:50:25.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 229f3e2d-0016-4050-a469-944d56435c4c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58072024-12-14T04:50:25.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 229f3e2d-0016-4050-a469-944d56435c4c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58082024-12-14T04:50:25.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2222 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 229f3e2d-0016-4050-a469-944d56435c4c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58092024-12-14T04:50:26.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7e1a1e61-7777-48dc-89b2-90bfa1cb2650 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58102024-12-14T04:50:26.598ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7e1a1e61-7777-48dc-89b2-90bfa1cb2650 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58112024-12-14T04:50:26.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2386 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7e1a1e61-7777-48dc-89b2-90bfa1cb2650 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58122024-12-14T04:50:26.606ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:52666
58132024-12-14T04:50:26.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52666 req_id = 47cdd819-2ad0-49af-b0ed-d061f8d2f329 roles = RoleSet { roles: {} } uri = /metrics/producers
58142024-12-14T04:50:26.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52666 req_id = 47cdd819-2ad0-49af-b0ed-d061f8d2f329 resource = Database result = Ok(()) uri = /metrics/producers
58152024-12-14T04:50:26.623ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 83153bbf-d864-44c8-b8a5-770b4235fe7a
58162024-12-14T04:50:26.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 16536 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52666 req_id = 47cdd819-2ad0-49af-b0ed-d061f8d2f329 response_code = 201 uri = /metrics/producers
58172024-12-14T04:50:26.882ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 93 reason = Timeout
58182024-12-14T04:50:26.882ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58192024-12-14T04:50:26.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58202024-12-14T04:50:26.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 93 reason = Timeout
58212024-12-14T04:50:26.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58222024-12-14T04:50:26.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58232024-12-14T04:50:26.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58242024-12-14T04:50:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58252024-12-14T04:50:26.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58262024-12-14T04:50:26.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58272024-12-14T04:50:26.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58282024-12-14T04:50:26.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58292024-12-14T04:50:26.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58302024-12-14T04:50:26.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
58312024-12-14T04:50:26.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58322024-12-14T04:50:26.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58332024-12-14T04:50:26.909ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
58342024-12-14T04:50:26.909ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.322676ms iteration = 93 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
58352024-12-14T04:50:26.918ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58362024-12-14T04:50:26.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58372024-12-14T04:50:26.947ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.343603ms iteration = 93 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58382024-12-14T04:50:27.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d121d4be-d576-4d26-b239-e179ad657f1c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58392024-12-14T04:50:27.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d121d4be-d576-4d26-b239-e179ad657f1c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58402024-12-14T04:50:27.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2696 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d121d4be-d576-4d26-b239-e179ad657f1c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58412024-12-14T04:50:28.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f7fd8100-8e99-4259-aa9e-717fadc79667 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58422024-12-14T04:50:28.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f7fd8100-8e99-4259-aa9e-717fadc79667 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58432024-12-14T04:50:28.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2068 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f7fd8100-8e99-4259-aa9e-717fadc79667 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58442024-12-14T04:50:29.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8296f405-f0b2-4cd8-8afc-a6bc7c278f49 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58452024-12-14T04:50:29.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8296f405-f0b2-4cd8-8afc-a6bc7c278f49 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58462024-12-14T04:50:29.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2783 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8296f405-f0b2-4cd8-8afc-a6bc7c278f49 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58472024-12-14T04:50:30.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8bcf20cd-f542-43b1-82da-5289616f4f2e roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58482024-12-14T04:50:30.688ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8bcf20cd-f542-43b1-82da-5289616f4f2e resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58492024-12-14T04:50:30.690ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2458 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 8bcf20cd-f542-43b1-82da-5289616f4f2e response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58502024-12-14T04:50:31.709ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b1fbf4db-8a5f-4017-91da-6b14ce09a840 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58512024-12-14T04:50:31.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b1fbf4db-8a5f-4017-91da-6b14ce09a840 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58522024-12-14T04:50:31.711ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2573 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = b1fbf4db-8a5f-4017-91da-6b14ce09a840 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58532024-12-14T04:50:32.731ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bc43c460-5e3e-445c-96ef-e7ddb4e5448a roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58542024-12-14T04:50:32.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bc43c460-5e3e-445c-96ef-e7ddb4e5448a resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58552024-12-14T04:50:32.734ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2418 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = bc43c460-5e3e-445c-96ef-e7ddb4e5448a response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58562024-12-14T04:50:32.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = f616ab30-2c40-46d9-96c4-efb45b21da8e roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
58572024-12-14T04:50:32.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = f616ab30-2c40-46d9-96c4-efb45b21da8e resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
58582024-12-14T04:50:32.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2444 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = f616ab30-2c40-46d9-96c4-efb45b21da8e response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
58592024-12-14T04:50:32.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = d83f02f1-50f5-4ff5-a47b-3503c4002c3b roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
58602024-12-14T04:50:32.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = d83f02f1-50f5-4ff5-a47b-3503c4002c3b resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
58612024-12-14T04:50:32.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2453 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = d83f02f1-50f5-4ff5-a47b-3503c4002c3b response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
58622024-12-14T04:50:32.785ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1462 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = e97685fb-bd30-4b77-9f8a-534471d68ca5 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
58632024-12-14T04:50:33.756ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d02ca30b-c0a9-4398-a5ac-5bfc905e398c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58642024-12-14T04:50:33.756ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d02ca30b-c0a9-4398-a5ac-5bfc905e398c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58652024-12-14T04:50:33.758ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2056 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = d02ca30b-c0a9-4398-a5ac-5bfc905e398c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58662024-12-14T04:50:34.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 49ed8ae9-0aed-4251-982c-a41e10e33e23 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58672024-12-14T04:50:34.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 49ed8ae9-0aed-4251-982c-a41e10e33e23 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58682024-12-14T04:50:34.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2388 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 49ed8ae9-0aed-4251-982c-a41e10e33e23 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58692024-12-14T04:50:35.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 47b6afba-9304-48a8-bee7-886ee7df76c2 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58702024-12-14T04:50:35.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 47b6afba-9304-48a8-bee7-886ee7df76c2 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58712024-12-14T04:50:35.802ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2821 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 47b6afba-9304-48a8-bee7-886ee7df76c2 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58722024-12-14T04:50:36.822ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 24a5fc4d-1252-4ce5-bd29-de76afc5c14d roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58732024-12-14T04:50:36.822ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 24a5fc4d-1252-4ce5-bd29-de76afc5c14d resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58742024-12-14T04:50:36.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1922 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 24a5fc4d-1252-4ce5-bd29-de76afc5c14d response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
58752024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 32 reason = Timeout
58762024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 32 reason = Timeout
58772024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 32 reason = Timeout
58782024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58792024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
58802024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
58812024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 32 reason = Timeout
58822024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
58832024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58842024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 32 reason = Timeout
58852024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 32 reason = Timeout
58862024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
58872024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58882024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 32 reason = Timeout
58892024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
58902024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 94 reason = Timeout
58912024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58922024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
58932024-12-14T04:50:36.883ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
58942024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58952024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 32 reason = Timeout
58962024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
58972024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58982024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 32 reason = Timeout
58992024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59002024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 32 reason = Timeout
59012024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59022024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 32 reason = Timeout
59032024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59042024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 33 reason = Timeout
59052024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59062024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
59072024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59082024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59092024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59102024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59112024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59122024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
59132024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
59142024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
59152024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59162024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59172024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59182024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59192024-12-14T04:50:36.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59202024-12-14T04:50:36.884ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 94 reason = Timeout
59212024-12-14T04:50:36.884ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 32 reason = Timeout
59222024-12-14T04:50:36.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59232024-12-14T04:50:36.884ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59242024-12-14T04:50:36.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59252024-12-14T04:50:36.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59262024-12-14T04:50:36.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59272024-12-14T04:50:36.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59282024-12-14T04:50:36.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59292024-12-14T04:50:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59302024-12-14T04:50:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59312024-12-14T04:50:36.887ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59322024-12-14T04:50:36.887ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.137938ms iteration = 32 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
59332024-12-14T04:50:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59342024-12-14T04:50:36.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59352024-12-14T04:50:36.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59362024-12-14T04:50:36.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59372024-12-14T04:50:36.890ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1610, tv_nsec: 986203449 } })
59382024-12-14T04:50:36.890ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
59392024-12-14T04:50:36.890ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1610, tv_nsec: 986863446 } })
59402024-12-14T04:50:36.890ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
59412024-12-14T04:50:36.891ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59422024-12-14T04:50:36.891ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
59432024-12-14T04:50:36.891ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59442024-12-14T04:50:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59452024-12-14T04:50:36.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59462024-12-14T04:50:36.892ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59472024-12-14T04:50:36.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59482024-12-14T04:50:36.893ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 10.35702ms iteration = 94 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
59492024-12-14T04:50:36.894ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59502024-12-14T04:50:36.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59512024-12-14T04:50:36.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
59522024-12-14T04:50:36.894ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59532024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59542024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
59552024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59562024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59572024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
59582024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59592024-12-14T04:50:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59602024-12-14T04:50:36.897ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59612024-12-14T04:50:36.897ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59622024-12-14T04:50:36.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59632024-12-14T04:50:36.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59642024-12-14T04:50:36.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59652024-12-14T04:50:36.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59662024-12-14T04:50:36.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59672024-12-14T04:50:36.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59682024-12-14T04:50:36.900ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59692024-12-14T04:50:36.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59702024-12-14T04:50:36.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59712024-12-14T04:50:36.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59722024-12-14T04:50:36.912ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 29.295224ms iteration = 32 status = {"errors":[],"records_set_to_done":[]}
59732024-12-14T04:50:36.922ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59742024-12-14T04:50:36.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59752024-12-14T04:50:36.923ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59762024-12-14T04:50:36.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59772024-12-14T04:50:36.925ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 42.372264ms iteration = 32 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
59782024-12-14T04:50:36.927ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59792024-12-14T04:50:36.928ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59802024-12-14T04:50:36.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59812024-12-14T04:50:36.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
59822024-12-14T04:50:36.929ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59832024-12-14T04:50:36.929ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 46.04806ms iteration = 32 status = {"errors":[],"garbage_collect_requested":[]}
59842024-12-14T04:50:36.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59852024-12-14T04:50:36.930ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59862024-12-14T04:50:36.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59872024-12-14T04:50:36.931ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59882024-12-14T04:50:36.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59892024-12-14T04:50:36.931ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-908d035f-c40e-4f3a-a9d9-68bebca1a6f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1611, tv_nsec: 27918716 } })
59902024-12-14T04:50:36.932ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
59912024-12-14T04:50:36.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59922024-12-14T04:50:36.942ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59932024-12-14T04:50:36.942ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
59942024-12-14T04:50:36.942ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59952024-12-14T04:50:36.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59962024-12-14T04:50:36.943ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
59972024-12-14T04:50:36.943ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 58.817053ms iteration = 32 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
59982024-12-14T04:50:36.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
59992024-12-14T04:50:36.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60002024-12-14T04:50:36.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60012024-12-14T04:50:36.958ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 75.824036ms iteration = 32 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
60022024-12-14T04:50:36.959ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
60032024-12-14T04:50:36.959ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60042024-12-14T04:50:36.959ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
60052024-12-14T04:50:36.959ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.20444ms iteration = 94 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
60062024-12-14T04:50:36.959ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c3e5a23-26fb-4ad6-9c7c-29030dd4bcce", "content-length": "26", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60072024-12-14T04:50:36.959ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60082024-12-14T04:50:36.961ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60092024-12-14T04:50:36.967ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60102024-12-14T04:50:36.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 908d035f-c40e-4f3a-a9d9-68bebca1a6f9, lookup_type: ById(908d035f-c40e-4f3a-a9d9-68bebca1a6f9) } result = Ok(())
60112024-12-14T04:50:36.969ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60122024-12-14T04:50:36.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60132024-12-14T04:50:36.971ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 87.965272ms iteration = 32 status = 0
60142024-12-14T04:50:36.971ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60152024-12-14T04:50:36.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60162024-12-14T04:50:36.978ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
60172024-12-14T04:50:36.979ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60182024-12-14T04:50:36.979ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eec84e6-5780-44da-bb0c-7ba7dcbbbe08", "content-length": "26", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60192024-12-14T04:50:36.979ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60202024-12-14T04:50:36.980ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60212024-12-14T04:50:36.985ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60222024-12-14T04:50:36.985ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
60232024-12-14T04:50:36.985ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 102.170372ms iteration = 32 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
60242024-12-14T04:50:37.002ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60252024-12-14T04:50:37.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60262024-12-14T04:50:37.009ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 126.336962ms iteration = 36 status = {"physical_disks_added":0}
60272024-12-14T04:50:37.021ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 zone_address = fd00:1122:3344:101::2
60282024-12-14T04:50:37.021ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60292024-12-14T04:50:37.021ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2954441a-41f9-46a3-8f1f-86e30dfe98c2", "content-length": "26", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60302024-12-14T04:50:37.021ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60312024-12-14T04:50:37.023ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
60322024-12-14T04:50:37.040ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
60332024-12-14T04:50:37.041ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f25c32-1532-4e91-b454-fb6017c39e2e", "content-length": "2", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60342024-12-14T04:50:37.042ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
60352024-12-14T04:50:37.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
60362024-12-14T04:50:37.045ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
60372024-12-14T04:50:37.045ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60382024-12-14T04:50:37.046ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "04394d54-bb1d-41fc-b88a-25128f8679b5", "content-length": "372", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60392024-12-14T04:50:37.046ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60402024-12-14T04:50:37.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60412024-12-14T04:50:37.048ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 165.165484ms iteration = 32 status = null
60422024-12-14T04:50:37.050ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 167.32364ms iteration = 32 status = {}
60432024-12-14T04:50:37.054ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60442024-12-14T04:50:37.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60452024-12-14T04:50:37.056ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60462024-12-14T04:50:37.058ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60472024-12-14T04:50:37.058ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60482024-12-14T04:50:37.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60492024-12-14T04:50:37.059ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60502024-12-14T04:50:37.061ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60512024-12-14T04:50:37.061ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60522024-12-14T04:50:37.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60532024-12-14T04:50:37.075ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 192.125546ms iteration = 32 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
60542024-12-14T04:50:37.089ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
60552024-12-14T04:50:37.089ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
60562024-12-14T04:50:37.089ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9e59552d-2e81-47e9-b3d3-9b9ff7eea48f", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60572024-12-14T04:50:37.089ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60582024-12-14T04:50:37.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60592024-12-14T04:50:37.092ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60602024-12-14T04:50:37.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60612024-12-14T04:50:37.095ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60622024-12-14T04:50:37.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60632024-12-14T04:50:37.099ZINFOnexus (BackgroundTasks): VPC route manager sled 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
60642024-12-14T04:50:37.099ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60652024-12-14T04:50:37.099ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "bb87e1ad-aae3-4293-a25c-4c36d71c7a0e", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60662024-12-14T04:50:37.101ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 218.460173ms iteration = 34 status = {}
60672024-12-14T04:50:37.131ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60682024-12-14T04:50:37.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60692024-12-14T04:50:37.133ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 62fedcae-1d4d-4664-9451-c8403405e0ea, rack_id: 10e5bd50-f747-45b4-965d-d177b41ca124, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(8fefc9a2-b26c-460d-9edf-47dafa9a8c1c) } rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
60702024-12-14T04:50:37.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60712024-12-14T04:50:37.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60722024-12-14T04:50:37.155ZDEBGnexus (BackgroundTasks): client request SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
60732024-12-14T04:50:37.156ZDEBGnexus (BackgroundTasks): client response SledAgent = 908d035f-c40e-4f3a-a9d9-68bebca1a6f9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d626777-e858-4cd2-abdc-cb66cbf6b6ee", "content-length": "691", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60742024-12-14T04:50:37.156ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
60752024-12-14T04:50:37.156ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
60762024-12-14T04:50:37.156ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
60772024-12-14T04:50:37.156ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 273.61994ms iteration = 33 status = {}
60782024-12-14T04:50:37.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 55b63b38-f7be-4026-be6b-569fb037ab79 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
60792024-12-14T04:50:37.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 55b63b38-f7be-4026-be6b-569fb037ab79 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
60802024-12-14T04:50:37.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3185 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:56862 req_id = 55b63b38-f7be-4026-be6b-569fb037ab79 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
60812024-12-14T04:50:37.165ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
60822024-12-14T04:50:37.166ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a2473c3-51df-4f40-81ab-3609c1047b96", "content-length": "134", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60832024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
60842024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 8fefc9a2-b26c-460d-9edf-47dafa9a8c1c
60852024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
60862024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
60872024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60882024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60892024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60902024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
60912024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60922024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
60932024-12-14T04:50:37.166ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124
60942024-12-14T04:50:37.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60952024-12-14T04:50:37.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60962024-12-14T04:50:37.169ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
60972024-12-14T04:50:37.169ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2914e19d-7257-4dd6-ac43-b23055978952", "content-length": "2", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
60982024-12-14T04:50:37.169ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
60992024-12-14T04:50:37.169ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cd858dc-7756-4d3a-991c-0a297841771c", "content-length": "52", "date": "Sat, 14 Dec 2024 04:50:36 GMT"} })
61002024-12-14T04:50:37.169ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch_location = Switch0
61012024-12-14T04:50:37.169ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
61022024-12-14T04:50:37.264ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d7011d28-2a1f-4e6b-9539-9294066d12f7", "date": "Sat, 14 Dec 2024 04:50:37 GMT"} })
61032024-12-14T04:50:37.264ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 switch = Switch0
61042024-12-14T04:50:37.264ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
61052024-12-14T04:50:37.264ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 10e5bd50-f747-45b4-965d-d177b41ca124 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dfde2dd-577a-4a14-b6f6-0bd94e31ebab", "content-length": "682", "date": "Sat, 14 Dec 2024 04:50:37 GMT"} })
61062024-12-14T04:50:37.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61072024-12-14T04:50:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61082024-12-14T04:50:37.268ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61092024-12-14T04:50:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61102024-12-14T04:50:37.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61112024-12-14T04:50:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61122024-12-14T04:50:37.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61132024-12-14T04:50:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61142024-12-14T04:50:37.280ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 397.296343ms iteration = 32 status = {}
61152024-12-14T04:50:37.827ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:65059
61162024-12-14T04:50:37.827ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65059 req_id = cc47ebb3-3892-4f2c-9ae8-41b4d9272580 roles = RoleSet { roles: {} } uri = /metrics/producers
61172024-12-14T04:50:37.827ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65059 req_id = cc47ebb3-3892-4f2c-9ae8-41b4d9272580 resource = Database result = Ok(()) uri = /metrics/producers
61182024-12-14T04:50:37.842ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 717229ef-6d10-4703-9dfa-5bad0367f106 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61192024-12-14T04:50:37.843ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 717229ef-6d10-4703-9dfa-5bad0367f106 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61202024-12-14T04:50:37.845ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2294 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 717229ef-6d10-4703-9dfa-5bad0367f106 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61212024-12-14T04:50:37.872ZINFOnexus: assigned collector to new producer collector_id = 8198feca-72a9-4d51-9c7f-345241658de0 file = nexus/src/app/oximeter.rs:76 producer_id = 7c8e60d1-cbc0-4976-b68e-162aaf207501
61222024-12-14T04:50:37.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45220 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65059 req_id = cc47ebb3-3892-4f2c-9ae8-41b4d9272580 response_code = 201 uri = /metrics/producers
61232024-12-14T04:50:38.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3ebfe8be-e23a-4593-9315-9f99dfd5d749 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61242024-12-14T04:50:38.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3ebfe8be-e23a-4593-9315-9f99dfd5d749 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61252024-12-14T04:50:38.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1880 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3ebfe8be-e23a-4593-9315-9f99dfd5d749 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61262024-12-14T04:50:39.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f260aba5-c5f2-4c5a-83c6-11c8cec2d28b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61272024-12-14T04:50:39.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f260aba5-c5f2-4c5a-83c6-11c8cec2d28b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61282024-12-14T04:50:39.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2533 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = f260aba5-c5f2-4c5a-83c6-11c8cec2d28b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61292024-12-14T04:50:40.908ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9704ef57-92ec-4ff8-9243-6ad2196271a9 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61302024-12-14T04:50:40.908ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9704ef57-92ec-4ff8-9243-6ad2196271a9 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61312024-12-14T04:50:40.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2240 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9704ef57-92ec-4ff8-9243-6ad2196271a9 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61322024-12-14T04:50:41.929ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 749c957e-81f7-4d47-a5c7-f1171e883aaa roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61332024-12-14T04:50:41.930ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 749c957e-81f7-4d47-a5c7-f1171e883aaa resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61342024-12-14T04:50:41.932ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2835 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 749c957e-81f7-4d47-a5c7-f1171e883aaa response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61352024-12-14T04:50:42.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1315 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = aea3c7dc-e64d-4b3d-aaea-d99e5207ad86 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
61362024-12-14T04:50:42.952ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e9d46194-6e57-4358-a8a3-c5bb479d277f roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61372024-12-14T04:50:42.952ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e9d46194-6e57-4358-a8a3-c5bb479d277f resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61382024-12-14T04:50:42.954ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2254 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = e9d46194-6e57-4358-a8a3-c5bb479d277f response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61392024-12-14T04:50:43.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34d841d0-d99f-41b1-802f-ec708b1a572c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61402024-12-14T04:50:43.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34d841d0-d99f-41b1-802f-ec708b1a572c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61412024-12-14T04:50:43.976ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2189 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 34d841d0-d99f-41b1-802f-ec708b1a572c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61422024-12-14T04:50:44.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37b107ec-b819-4e9a-a120-53c6e2be6b32 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61432024-12-14T04:50:44.996ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37b107ec-b819-4e9a-a120-53c6e2be6b32 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61442024-12-14T04:50:44.998ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2586 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 37b107ec-b819-4e9a-a120-53c6e2be6b32 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61452024-12-14T04:50:46.018ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7d787997-cf97-435d-a7e5-d25f42d56e8b roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61462024-12-14T04:50:46.019ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7d787997-cf97-435d-a7e5-d25f42d56e8b resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61472024-12-14T04:50:46.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2543 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 7d787997-cf97-435d-a7e5-d25f42d56e8b response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61482024-12-14T04:50:46.883ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 95 reason = Timeout
61492024-12-14T04:50:46.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61502024-12-14T04:50:46.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61512024-12-14T04:50:46.883ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 95 reason = Timeout
61522024-12-14T04:50:46.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61532024-12-14T04:50:46.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61542024-12-14T04:50:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61552024-12-14T04:50:46.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61562024-12-14T04:50:46.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61572024-12-14T04:50:46.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.415365ms iteration = 95 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61582024-12-14T04:50:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
61592024-12-14T04:50:46.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61602024-12-14T04:50:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61612024-12-14T04:50:46.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61622024-12-14T04:50:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61632024-12-14T04:50:46.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61642024-12-14T04:50:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 274dd1d6-4a85-4a29-9780-ddbf1eedc832, lookup_type: ById(274dd1d6-4a85-4a29-9780-ddbf1eedc832) } result = Ok(())
61652024-12-14T04:50:46.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61662024-12-14T04:50:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61672024-12-14T04:50:46.905ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 original_time_created = 2024-12-14 04:35:11.841845 UTC target_id = 274dd1d6-4a85-4a29-9780-ddbf1eedc832 time_created = 2024-12-14 04:35:11.841845 UTC
61682024-12-14T04:50:46.905ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.025354ms iteration = 95 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"274dd1d6-4a85-4a29-9780-ddbf1eedc832","time_created":"2024-12-14T04:35:11.841845Z"}
61692024-12-14T04:50:47.042ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11dd1e97-7f2e-4328-b154-fffc2b3366b3 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61702024-12-14T04:50:47.043ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11dd1e97-7f2e-4328-b154-fffc2b3366b3 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61712024-12-14T04:50:47.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2323 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 11dd1e97-7f2e-4328-b154-fffc2b3366b3 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61722024-12-14T04:50:47.771ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 927edd02-c2e7-4f18-a3ab-fff0e451ddfb roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
61732024-12-14T04:50:47.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 927edd02-c2e7-4f18-a3ab-fff0e451ddfb resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
61742024-12-14T04:50:47.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2241 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 927edd02-c2e7-4f18-a3ab-fff0e451ddfb response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?sort_by=id_ascending
61752024-12-14T04:50:47.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 9538fba0-b788-424d-9ad8-fc72a589d707 roles = RoleSet { roles: {} } uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
61762024-12-14T04:50:47.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 9538fba0-b788-424d-9ad8-fc72a589d707 resource = Database result = Ok(()) uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
61772024-12-14T04:50:47.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2593 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41967 req_id = 9538fba0-b788-424d-9ad8-fc72a589d707 response_code = 200 uri = /metrics/collectors/8198feca-72a9-4d51-9c7f-345241658de0/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlMzZmZDA5MS1lZmNlLTRlYTUtOWU3My1hOWJjNGNiN2E1NDkifX0%3D
61782024-12-14T04:50:48.065ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3398d1f3-016a-4aaa-8733-02b22a4d0448 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61792024-12-14T04:50:48.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3398d1f3-016a-4aaa-8733-02b22a4d0448 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61802024-12-14T04:50:48.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2040 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3398d1f3-016a-4aaa-8733-02b22a4d0448 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61812024-12-14T04:50:49.087ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ab000f4a-badb-4374-9450-9e7dba9d1264 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61822024-12-14T04:50:49.088ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ab000f4a-badb-4374-9450-9e7dba9d1264 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61832024-12-14T04:50:49.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2385 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = ab000f4a-badb-4374-9450-9e7dba9d1264 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61842024-12-14T04:50:50.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3fa2c846-c4d8-496f-bde8-1cb29b0e2c23 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61852024-12-14T04:50:50.110ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3fa2c846-c4d8-496f-bde8-1cb29b0e2c23 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61862024-12-14T04:50:50.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2444 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 3fa2c846-c4d8-496f-bde8-1cb29b0e2c23 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61872024-12-14T04:50:51.133ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4e5eac96-7c88-44f8-a310-159f4285e12c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61882024-12-14T04:50:51.133ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4e5eac96-7c88-44f8-a310-159f4285e12c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61892024-12-14T04:50:51.135ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2147 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 4e5eac96-7c88-44f8-a310-159f4285e12c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61902024-12-14T04:50:52.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0f1de182-e673-47c3-927c-f7ab8be6ca8c roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61912024-12-14T04:50:52.155ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0f1de182-e673-47c3-927c-f7ab8be6ca8c resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61922024-12-14T04:50:52.157ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2283 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 0f1de182-e673-47c3-927c-f7ab8be6ca8c response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61932024-12-14T04:50:52.784ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1467 local_addr = [fd00:1122:3344:101::c]:35272 method = GET remote_addr = [fd00:1122:3344:101::d]:39316 req_id = 9e3e78f6-9e1a-4279-8900-c520b42ef4a1 response_code = 200 uri = /a2394d1d-754d-4ed4-8218-d02ec0d792df
61942024-12-14T04:50:53.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9a00c6f5-e717-40ee-807f-8985ae529490 roles = RoleSet { roles: {} } uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61952024-12-14T04:50:53.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9a00c6f5-e717-40ee-807f-8985ae529490 resource = Database result = Ok(()) uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9
61962024-12-14T04:50:53.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2850 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:51885 req_id = 9a00c6f5-e717-40ee-807f-8985ae529490 response_code = 200 uri = /probes/908d035f-c40e-4f3a-a9d9-68bebca1a6f9