Max level shown:
12025-01-15T02:30:02.019ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cd75108-1492-4ba1-8d86-c3f83bd00679 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
22025-01-15T02:30:02.020ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cd75108-1492-4ba1-8d86-c3f83bd00679 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
32025-01-15T02:30:02.022ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2671 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cd75108-1492-4ba1-8d86-c3f83bd00679 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
42025-01-15T02:30:03.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4635b64f-247a-4366-92c9-f96b21740270 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52025-01-15T02:30:03.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4635b64f-247a-4366-92c9-f96b21740270 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62025-01-15T02:30:03.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4635b64f-247a-4366-92c9-f96b21740270 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72025-01-15T02:30:04.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d58d38b8-2305-4eda-a6b2-55154b506d49 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
82025-01-15T02:30:04.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d58d38b8-2305-4eda-a6b2-55154b506d49 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
92025-01-15T02:30:04.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2435 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d58d38b8-2305-4eda-a6b2-55154b506d49 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
102025-01-15T02:30:05.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1998e433-c465-48c9-a801-4ee023ec786f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
112025-01-15T02:30:05.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1998e433-c465-48c9-a801-4ee023ec786f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
122025-01-15T02:30:05.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2310 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1998e433-c465-48c9-a801-4ee023ec786f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
132025-01-15T02:30:05.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 494 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 1d901751-36ee-4fca-b16b-f9f193bf040d response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
142025-01-15T02:30:06.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88ff7eee-87b7-4709-8816-f5872df46efd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
152025-01-15T02:30:06.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88ff7eee-87b7-4709-8816-f5872df46efd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
162025-01-15T02:30:06.119ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2272 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88ff7eee-87b7-4709-8816-f5872df46efd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
172025-01-15T02:30:07.140ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 80e54e08-cbeb-4e96-b83b-5cea30c3182b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
182025-01-15T02:30:07.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 80e54e08-cbeb-4e96-b83b-5cea30c3182b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
192025-01-15T02:30:07.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1641 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 80e54e08-cbeb-4e96-b83b-5cea30c3182b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
202025-01-15T02:30:08.161ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9fe8c853-3c27-4e85-bef1-9037d17fcc8f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
212025-01-15T02:30:08.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9fe8c853-3c27-4e85-bef1-9037d17fcc8f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
222025-01-15T02:30:08.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2000 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9fe8c853-3c27-4e85-bef1-9037d17fcc8f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
232025-01-15T02:30:09.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe8460eb-e0d4-4dbe-a3ad-a2f08977df5a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
242025-01-15T02:30:09.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe8460eb-e0d4-4dbe-a3ad-a2f08977df5a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
252025-01-15T02:30:09.185ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2136 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe8460eb-e0d4-4dbe-a3ad-a2f08977df5a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
262025-01-15T02:30:10.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f512d703-f072-4f8d-96fb-00ea256c12dd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
272025-01-15T02:30:10.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f512d703-f072-4f8d-96fb-00ea256c12dd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
282025-01-15T02:30:10.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2292 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f512d703-f072-4f8d-96fb-00ea256c12dd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
292025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
302025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
312025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
322025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
332025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
342025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
352025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
362025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
372025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
382025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
392025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
402025-01-15T02:30:10.507ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:20:10.507379722 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
412025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
422025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
432025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
442025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
452025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
462025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
472025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
482025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
492025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
502025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
512025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
522025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
532025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
542025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
552025-01-15T02:30:10.507ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
562025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
572025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
582025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
592025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
602025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
612025-01-15T02:30:10.507ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
622025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
632025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
642025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
652025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
662025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
672025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
682025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
692025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
702025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
712025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
722025-01-15T02:30:10.507ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
732025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
742025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
752025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.83\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
762025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
772025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
782025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
792025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
802025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
812025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
822025-01-15T02:30:10.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
832025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
842025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
852025-01-15T02:30:10.508ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3497, tv_nsec: 342167408 } })
862025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
872025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
882025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
892025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
902025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
912025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
922025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
932025-01-15T02:30:10.508ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3497, tv_nsec: 342270700 } })
942025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
952025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
962025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
972025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
982025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
992025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1002025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1012025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1022025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1032025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1042025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1052025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1062025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
1072025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1082025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
1092025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1102025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1112025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
1122025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1132025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1142025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
1152025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1162025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
1172025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1182025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
1192025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1202025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1212025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1222025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1232025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.261745ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
1242025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1252025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1262025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1272025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
1282025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.445379ms iteration = 4 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
1292025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1302025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1312025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1322025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1332025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1342025-01-15T02:30:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1352025-01-15T02:30:10.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362025-01-15T02:30:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.127094ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1372025-01-15T02:30:10.510ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:20:10.507379722 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1382025-01-15T02:30:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.290959ms iteration = 4 status = {"expiration":"2025-01-15T02:20:10.507379722Z","pruned":[]}
1392025-01-15T02:30:10.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1402025-01-15T02:30:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1412025-01-15T02:30:10.511ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3497, tv_nsec: 345782995 } })
1422025-01-15T02:30:10.511ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1432025-01-15T02:30:10.512ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3497, tv_nsec: 346366391 } })
1442025-01-15T02:30:10.512ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1452025-01-15T02:30:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1462025-01-15T02:30:10.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1472025-01-15T02:30:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1482025-01-15T02:30:10.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1492025-01-15T02:30:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1502025-01-15T02:30:10.516ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1512025-01-15T02:30:10.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1522025-01-15T02:30:10.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1532025-01-15T02:30:10.518ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1542025-01-15T02:30:10.518ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1552025-01-15T02:30:10.519ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1562025-01-15T02:30:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1572025-01-15T02:30:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1582025-01-15T02:30:10.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1592025-01-15T02:30:10.527ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602025-01-15T02:30:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1612025-01-15T02:30:10.527ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1622025-01-15T02:30:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1632025-01-15T02:30:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
1642025-01-15T02:30:10.529ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1652025-01-15T02:30:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1662025-01-15T02:30:10.530ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1672025-01-15T02:30:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1682025-01-15T02:30:10.532ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1692025-01-15T02:30:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1702025-01-15T02:30:10.536ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 28.411117ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
1712025-01-15T02:30:10.549ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1722025-01-15T02:30:10.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1732025-01-15T02:30:10.552ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1742025-01-15T02:30:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1752025-01-15T02:30:10.557ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1762025-01-15T02:30:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1772025-01-15T02:30:10.561ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1782025-01-15T02:30:10.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1792025-01-15T02:30:10.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1802025-01-15T02:30:10.564ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1812025-01-15T02:30:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1822025-01-15T02:30:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1832025-01-15T02:30:10.564ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1842025-01-15T02:30:10.564ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1852025-01-15T02:30:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1862025-01-15T02:30:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1872025-01-15T02:30:10.566ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1882025-01-15T02:30:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1892025-01-15T02:30:10.569ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1902025-01-15T02:30:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1912025-01-15T02:30:10.571ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1922025-01-15T02:30:10.572ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1932025-01-15T02:30:10.572ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1942025-01-15T02:30:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1952025-01-15T02:30:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1962025-01-15T02:30:10.574ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1972025-01-15T02:30:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1982025-01-15T02:30:10.575ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1992025-01-15T02:30:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2002025-01-15T02:30:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2012025-01-15T02:30:10.577ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2022025-01-15T02:30:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2032025-01-15T02:30:10.581ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3497, tv_nsec: 415276151 } })
2042025-01-15T02:30:10.581ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2052025-01-15T02:30:10.581ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2062025-01-15T02:30:10.581ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
2072025-01-15T02:30:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2082025-01-15T02:30:10.582ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2092025-01-15T02:30:10.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2102025-01-15T02:30:10.582ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2112025-01-15T02:30:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2122025-01-15T02:30:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2132025-01-15T02:30:10.584ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2142025-01-15T02:30:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2152025-01-15T02:30:10.584ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2162025-01-15T02:30:10.585ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2172025-01-15T02:30:10.585ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2182025-01-15T02:30:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 76.687329ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2192025-01-15T02:30:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2202025-01-15T02:30:10.585ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2212025-01-15T02:30:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2222025-01-15T02:30:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2232025-01-15T02:30:10.587ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2242025-01-15T02:30:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
2252025-01-15T02:30:10.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2262025-01-15T02:30:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2272025-01-15T02:30:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2282025-01-15T02:30:10.589ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2292025-01-15T02:30:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2302025-01-15T02:30:10.591ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2312025-01-15T02:30:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2322025-01-15T02:30:10.592ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.950635ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
2332025-01-15T02:30:10.592ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2342025-01-15T02:30:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2352025-01-15T02:30:10.592ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2362025-01-15T02:30:10.593ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 84.912591ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
2372025-01-15T02:30:10.595ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2382025-01-15T02:30:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2392025-01-15T02:30:10.596ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2402025-01-15T02:30:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2412025-01-15T02:30:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2422025-01-15T02:30:10.598ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2432025-01-15T02:30:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2442025-01-15T02:30:10.600ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 92.908907ms iteration = 3 status = {"nsuccess":5}
2452025-01-15T02:30:10.601ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 93.096772ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2462025-01-15T02:30:10.603ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2472025-01-15T02:30:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2482025-01-15T02:30:10.606ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2492025-01-15T02:30:10.606ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2502025-01-15T02:30:10.607ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "62f8a4b4-ff39-4397-8ec7-b0bb98f897f7", "content-length": "26", "date": "Wed, 15 Jan 2025 02:30:09 GMT"} })
2512025-01-15T02:30:10.607ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2522025-01-15T02:30:10.608ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2532025-01-15T02:30:10.610ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2542025-01-15T02:30:10.613ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
2552025-01-15T02:30:10.613ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 106.431582ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
2562025-01-15T02:30:10.619ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2572025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2582025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2592025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2602025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2612025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2622025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2632025-01-15T02:30:10.620ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2642025-01-15T02:30:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2652025-01-15T02:30:10.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2662025-01-15T02:30:10.623ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2672025-01-15T02:30:10.623ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2682025-01-15T02:30:10.623ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2692025-01-15T02:30:10.623ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2702025-01-15T02:30:10.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2712025-01-15T02:30:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2722025-01-15T02:30:10.626ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2732025-01-15T02:30:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2742025-01-15T02:30:10.626ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2752025-01-15T02:30:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2762025-01-15T02:30:10.627ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2772025-01-15T02:30:10.627ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2782025-01-15T02:30:10.627ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2792025-01-15T02:30:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2802025-01-15T02:30:10.627ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2812025-01-15T02:30:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2822025-01-15T02:30:10.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2832025-01-15T02:30:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2842025-01-15T02:30:10.629ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2852025-01-15T02:30:10.629ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
2862025-01-15T02:30:10.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2872025-01-15T02:30:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2882025-01-15T02:30:10.631ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2892025-01-15T02:30:10.631ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2902025-01-15T02:30:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2912025-01-15T02:30:10.631ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2922025-01-15T02:30:10.632ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 124.548161ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2932025-01-15T02:30:10.632ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 125.095806ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2942025-01-15T02:30:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2952025-01-15T02:30:10.633ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2962025-01-15T02:30:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2972025-01-15T02:30:10.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2982025-01-15T02:30:10.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2992025-01-15T02:30:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3002025-01-15T02:30:10.637ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012025-01-15T02:30:10.638ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
3022025-01-15T02:30:10.638ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3032025-01-15T02:30:10.638ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
3042025-01-15T02:30:10.638ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 131.121848ms iteration = 4 status = {"generation":2}
3052025-01-15T02:30:10.638ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3062025-01-15T02:30:10.638ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
3072025-01-15T02:30:10.638ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 131.262152ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3082025-01-15T02:30:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
3092025-01-15T02:30:10.639ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3102025-01-15T02:30:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3112025-01-15T02:30:10.640ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3122025-01-15T02:30:10.642ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 134.90206ms iteration = 7 status = 0
3132025-01-15T02:30:10.642ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3142025-01-15T02:30:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3152025-01-15T02:30:10.645ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3162025-01-15T02:30:10.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3172025-01-15T02:30:10.647ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3182025-01-15T02:30:10.648ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3192025-01-15T02:30:10.648ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3202025-01-15T02:30:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3212025-01-15T02:30:10.649ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3222025-01-15T02:30:10.649ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 141.431117ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3232025-01-15T02:30:10.656ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3242025-01-15T02:30:10.656ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3252025-01-15T02:30:10.656ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7673e12-b21e-4b4c-80a1-26e75a234d46", "content-length": "26", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
3262025-01-15T02:30:10.656ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3272025-01-15T02:30:10.658ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3282025-01-15T02:30:10.658ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3292025-01-15T02:30:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3302025-01-15T02:30:10.661ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3312025-01-15T02:30:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3322025-01-15T02:30:10.671ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3332025-01-15T02:30:10.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3342025-01-15T02:30:10.672ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3352025-01-15T02:30:10.672ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3362025-01-15T02:30:10.672ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 164.889079ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3372025-01-15T02:30:10.682ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
3382025-01-15T02:30:10.682ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 175.144856ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3392025-01-15T02:30:10.687ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 179.609486ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3402025-01-15T02:30:10.700ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
3412025-01-15T02:30:10.700ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3422025-01-15T02:30:10.701ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f399f0d6-5f1f-4eaf-954e-c167964788f6", "content-length": "26", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
3432025-01-15T02:30:10.701ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3442025-01-15T02:30:10.702ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
3452025-01-15T02:30:10.707ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3462025-01-15T02:30:10.707ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3472025-01-15T02:30:10.707ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3482025-01-15T02:30:10.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3492025-01-15T02:30:10.708ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3502025-01-15T02:30:10.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3512025-01-15T02:30:10.712ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3522025-01-15T02:30:10.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3532025-01-15T02:30:10.715ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
3542025-01-15T02:30:10.715ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 207.770667ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3552025-01-15T02:30:10.717ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3562025-01-15T02:30:10.717ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3572025-01-15T02:30:10.717ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
3582025-01-15T02:30:10.717ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 209.966946ms iteration = 4 status = {"generation":1}
3592025-01-15T02:30:10.735ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3602025-01-15T02:30:10.735ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "32381122-1cdc-414d-8435-9f3aa9da7be4", "content-length": "1", "date": "Wed, 15 Jan 2025 02:30:09 GMT"} })
3612025-01-15T02:30:10.736ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 228.897957ms iteration = 9 status = {"physical_disks_added":0}
3622025-01-15T02:30:10.738ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3632025-01-15T02:30:10.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3642025-01-15T02:30:10.740ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 233.194333ms iteration = 7 status = null
3652025-01-15T02:30:10.745ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3662025-01-15T02:30:10.754ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3672025-01-15T02:30:10.754ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 246.529932ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3682025-01-15T02:30:10.778ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3692025-01-15T02:30:10.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3702025-01-15T02:30:10.784ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3712025-01-15T02:30:10.784ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9b4d6dec-1709-4e40-baff-f6f8d0310291", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3722025-01-15T02:30:10.784ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3732025-01-15T02:30:10.787ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 279.906013ms iteration = 7 status = {}
3742025-01-15T02:30:10.788ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5280fcd7-41a1-40e6-a79b-cae8538a8f6b", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3752025-01-15T02:30:10.788ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3762025-01-15T02:30:10.790ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3772025-01-15T02:30:10.790ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 283.47762ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3782025-01-15T02:30:10.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3792025-01-15T02:30:10.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3802025-01-15T02:30:10.807ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3812025-01-15T02:30:10.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3822025-01-15T02:30:10.808ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3832025-01-15T02:30:10.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3842025-01-15T02:30:10.810ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3852025-01-15T02:30:10.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3862025-01-15T02:30:10.813ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3872025-01-15T02:30:10.813ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3882025-01-15T02:30:10.814ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0b3b25b-6db2-4b6c-97ac-88a9b99e0328", "content-length": "372", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
3892025-01-15T02:30:10.814ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3902025-01-15T02:30:10.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3912025-01-15T02:30:10.831ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3922025-01-15T02:30:10.831ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "e68456f4-7676-44dc-80f5-37da33e3baba", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3932025-01-15T02:30:10.831ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3942025-01-15T02:30:10.833ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "08565ae2-73a1-4a3c-9c6d-6a64f1772dda", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3952025-01-15T02:30:10.833ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3962025-01-15T02:30:10.834ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5bec691e-1574-44db-9845-66b362bc6ba6", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3972025-01-15T02:30:10.834ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3982025-01-15T02:30:10.835ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3992025-01-15T02:30:10.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 328.435972ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4002025-01-15T02:30:10.857ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4012025-01-15T02:30:10.857ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4022025-01-15T02:30:10.857ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "df2d941a-777e-4b70-9096-004c8766c988", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4032025-01-15T02:30:10.857ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4042025-01-15T02:30:10.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4052025-01-15T02:30:10.860ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4062025-01-15T02:30:10.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4072025-01-15T02:30:10.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4082025-01-15T02:30:10.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4092025-01-15T02:30:10.867ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4102025-01-15T02:30:10.867ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4112025-01-15T02:30:10.868ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a216e6d6-6ba6-4a90-a76e-2e7cda2c0063", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4122025-01-15T02:30:10.869ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 361.992918ms iteration = 7 status = {}
4132025-01-15T02:30:10.896ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4142025-01-15T02:30:10.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4152025-01-15T02:30:10.915ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4162025-01-15T02:30:10.916ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e0212ef-4a1e-4154-8ad3-b802e7588c20", "content-length": "691", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4172025-01-15T02:30:10.916ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4182025-01-15T02:30:10.916ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4192025-01-15T02:30:10.916ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4202025-01-15T02:30:10.917ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
4212025-01-15T02:30:10.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4222025-01-15T02:30:10.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4232025-01-15T02:30:10.917ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 410.386085ms iteration = 7 status = {}
4242025-01-15T02:30:10.927ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4252025-01-15T02:30:10.927ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "61db6002-bc5e-4a9b-8a0a-4edada5419a9", "content-length": "134", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4262025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4272025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
4282025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4292025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4302025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4312025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4322025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4332025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4342025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4352025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4362025-01-15T02:30:10.927ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
4372025-01-15T02:30:10.927ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4382025-01-15T02:30:10.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4392025-01-15T02:30:10.929ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4402025-01-15T02:30:10.929ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c5a6fc5-e1b9-4087-b51f-d982e892877c", "content-length": "2", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4412025-01-15T02:30:10.929ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4422025-01-15T02:30:10.930ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf60f017-ff6c-4dbc-9d42-faced039f4ed", "content-length": "52", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4432025-01-15T02:30:10.930ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
4442025-01-15T02:30:10.930ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4452025-01-15T02:30:11.026ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f7b505f3-397c-4682-9baa-bb79f0e15df7", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4462025-01-15T02:30:11.026ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
4472025-01-15T02:30:11.026ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4482025-01-15T02:30:11.026ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "29a88133-b595-49dd-81d1-7ea1535dc8a2", "content-length": "682", "date": "Wed, 15 Jan 2025 02:30:10 GMT"} })
4492025-01-15T02:30:11.026ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4502025-01-15T02:30:11.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4512025-01-15T02:30:11.030ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4522025-01-15T02:30:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4532025-01-15T02:30:11.033ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4542025-01-15T02:30:11.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4552025-01-15T02:30:11.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4562025-01-15T02:30:11.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4572025-01-15T02:30:11.044ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 536.550559ms iteration = 7 status = {}
4582025-01-15T02:30:11.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d2d7b55b-95a1-416a-b8c7-9f42352e091d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4592025-01-15T02:30:11.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d2d7b55b-95a1-416a-b8c7-9f42352e091d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4602025-01-15T02:30:11.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2160 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d2d7b55b-95a1-416a-b8c7-9f42352e091d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4612025-01-15T02:30:12.336ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6452dbac-e3f5-4c4e-b9e6-1ae32670c960 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4622025-01-15T02:30:12.337ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6452dbac-e3f5-4c4e-b9e6-1ae32670c960 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4632025-01-15T02:30:12.338ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1773 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6452dbac-e3f5-4c4e-b9e6-1ae32670c960 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4642025-01-15T02:30:13.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c1ef8b1-c54a-4656-abdb-e2bf38de5eab roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4652025-01-15T02:30:13.359ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c1ef8b1-c54a-4656-abdb-e2bf38de5eab resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4662025-01-15T02:30:13.361ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2478 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c1ef8b1-c54a-4656-abdb-e2bf38de5eab response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4672025-01-15T02:30:14.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6048bd4b-b6cc-44dc-8a70-d05b07c99e0e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4682025-01-15T02:30:14.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6048bd4b-b6cc-44dc-8a70-d05b07c99e0e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4692025-01-15T02:30:14.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2512 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6048bd4b-b6cc-44dc-8a70-d05b07c99e0e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4702025-01-15T02:30:15.402ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23315a50-6f80-4fed-9233-d146cc4f7f84 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4712025-01-15T02:30:15.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23315a50-6f80-4fed-9233-d146cc4f7f84 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4722025-01-15T02:30:15.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23315a50-6f80-4fed-9233-d146cc4f7f84 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4732025-01-15T02:30:15.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 432 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 753c63ce-86d5-48c0-8e41-0eb05e484e42 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
4742025-01-15T02:30:16.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c621a2f-0f64-4b71-9c35-d92b44adc197 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4752025-01-15T02:30:16.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c621a2f-0f64-4b71-9c35-d92b44adc197 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4762025-01-15T02:30:16.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1890 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c621a2f-0f64-4b71-9c35-d92b44adc197 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4772025-01-15T02:30:17.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a21a2b0c-8081-4fab-8da8-fb749e06c816 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4782025-01-15T02:30:17.450ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a21a2b0c-8081-4fab-8da8-fb749e06c816 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4792025-01-15T02:30:17.451ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1787 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a21a2b0c-8081-4fab-8da8-fb749e06c816 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4802025-01-15T02:30:18.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f2bd97fb-d77a-47e4-aa0c-8ae116029875 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4812025-01-15T02:30:18.472ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f2bd97fb-d77a-47e4-aa0c-8ae116029875 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4822025-01-15T02:30:18.474ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2283 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f2bd97fb-d77a-47e4-aa0c-8ae116029875 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4832025-01-15T02:30:19.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4fe8b45c-4ad8-4a59-8ebb-7edd9f31594d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4842025-01-15T02:30:19.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4fe8b45c-4ad8-4a59-8ebb-7edd9f31594d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4852025-01-15T02:30:19.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2496 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4fe8b45c-4ad8-4a59-8ebb-7edd9f31594d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
4862025-01-15T02:30:20.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
4872025-01-15T02:30:20.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
4882025-01-15T02:30:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4892025-01-15T02:30:20.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4902025-01-15T02:30:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4912025-01-15T02:30:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4922025-01-15T02:30:20.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4932025-01-15T02:30:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4942025-01-15T02:30:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4952025-01-15T02:30:20.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.143865ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4962025-01-15T02:30:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4972025-01-15T02:30:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4982025-01-15T02:30:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4992025-01-15T02:30:20.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5002025-01-15T02:30:20.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5012025-01-15T02:30:20.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5022025-01-15T02:30:20.517ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e6fe60b4-1cd0-4d83-96c2-c7e45b2644ee roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5032025-01-15T02:30:20.517ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e6fe60b4-1cd0-4d83-96c2-c7e45b2644ee resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5042025-01-15T02:30:20.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
5052025-01-15T02:30:20.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5062025-01-15T02:30:20.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5072025-01-15T02:30:20.519ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1851 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e6fe60b4-1cd0-4d83-96c2-c7e45b2644ee response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5082025-01-15T02:30:20.529ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
5092025-01-15T02:30:20.529ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.137141ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
5102025-01-15T02:30:21.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d4cd742-3dc1-42ca-99a2-e1eb6432faa3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5112025-01-15T02:30:21.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d4cd742-3dc1-42ca-99a2-e1eb6432faa3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5122025-01-15T02:30:21.541ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2035 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d4cd742-3dc1-42ca-99a2-e1eb6432faa3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5132025-01-15T02:30:22.561ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d06b455-9e8c-4728-9f08-10ec978aa6ff roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5142025-01-15T02:30:22.562ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d06b455-9e8c-4728-9f08-10ec978aa6ff resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5152025-01-15T02:30:22.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1979 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d06b455-9e8c-4728-9f08-10ec978aa6ff response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5162025-01-15T02:30:23.584ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82c20042-b55e-49d5-bdf3-332e6558c15c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5172025-01-15T02:30:23.584ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82c20042-b55e-49d5-bdf3-332e6558c15c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5182025-01-15T02:30:23.586ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2435 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82c20042-b55e-49d5-bdf3-332e6558c15c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5192025-01-15T02:30:24.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cfdd1127-7d8d-4f80-83fc-955e50aac19c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5202025-01-15T02:30:24.606ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cfdd1127-7d8d-4f80-83fc-955e50aac19c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5212025-01-15T02:30:24.608ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2114 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cfdd1127-7d8d-4f80-83fc-955e50aac19c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5222025-01-15T02:30:25.628ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9bea714d-cb7b-4d42-a409-dcbc40d6569e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5232025-01-15T02:30:25.629ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9bea714d-cb7b-4d42-a409-dcbc40d6569e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5242025-01-15T02:30:25.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2419 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9bea714d-cb7b-4d42-a409-dcbc40d6569e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5252025-01-15T02:30:25.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 457 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = bdce7f21-8638-4628-8268-91615d43482e response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
5262025-01-15T02:30:26.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3714d7e2-4df8-48c8-8a21-3074f2bc32e2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5272025-01-15T02:30:26.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3714d7e2-4df8-48c8-8a21-3074f2bc32e2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5282025-01-15T02:30:26.652ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3714d7e2-4df8-48c8-8a21-3074f2bc32e2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5292025-01-15T02:30:27.672ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5d32390-85d5-406b-9e59-2949688cd1aa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5302025-01-15T02:30:27.672ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5d32390-85d5-406b-9e59-2949688cd1aa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5312025-01-15T02:30:27.674ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2410 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5d32390-85d5-406b-9e59-2949688cd1aa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5322025-01-15T02:30:28.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1a60b76a-44a0-4ccc-86a5-65e16bfc53ba roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5332025-01-15T02:30:28.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1a60b76a-44a0-4ccc-86a5-65e16bfc53ba resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5342025-01-15T02:30:28.696ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1979 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1a60b76a-44a0-4ccc-86a5-65e16bfc53ba response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5352025-01-15T02:30:29.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a071b6db-451c-4649-a96c-daef419d6d23 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5362025-01-15T02:30:29.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a071b6db-451c-4649-a96c-daef419d6d23 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5372025-01-15T02:30:29.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2233 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a071b6db-451c-4649-a96c-daef419d6d23 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5382025-01-15T02:30:30.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
5392025-01-15T02:30:30.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
5402025-01-15T02:30:30.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5412025-01-15T02:30:30.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5422025-01-15T02:30:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5432025-01-15T02:30:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5442025-01-15T02:30:30.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5452025-01-15T02:30:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5462025-01-15T02:30:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5472025-01-15T02:30:30.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.607558ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5482025-01-15T02:30:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5492025-01-15T02:30:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5502025-01-15T02:30:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5512025-01-15T02:30:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5522025-01-15T02:30:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5532025-01-15T02:30:30.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5542025-01-15T02:30:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
5552025-01-15T02:30:30.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5562025-01-15T02:30:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5572025-01-15T02:30:30.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
5582025-01-15T02:30:30.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.29871ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
5592025-01-15T02:30:30.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 42e4fae7-ca84-43ab-9d10-23d065ade8f5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5602025-01-15T02:30:30.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 42e4fae7-ca84-43ab-9d10-23d065ade8f5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5612025-01-15T02:30:30.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1558 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 42e4fae7-ca84-43ab-9d10-23d065ade8f5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5622025-01-15T02:30:31.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c19761b9-588b-4463-9c98-fe5cf7b5efeb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5632025-01-15T02:30:31.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c19761b9-588b-4463-9c98-fe5cf7b5efeb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5642025-01-15T02:30:31.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2747 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c19761b9-588b-4463-9c98-fe5cf7b5efeb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5652025-01-15T02:30:32.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fb431399-fe1c-4896-b94d-1f8b11b03119 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5662025-01-15T02:30:32.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fb431399-fe1c-4896-b94d-1f8b11b03119 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5672025-01-15T02:30:32.784ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2007 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fb431399-fe1c-4896-b94d-1f8b11b03119 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5682025-01-15T02:30:33.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 952ee3ce-fed9-41b6-a4da-92e0eed8b084 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5692025-01-15T02:30:33.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 952ee3ce-fed9-41b6-a4da-92e0eed8b084 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5702025-01-15T02:30:33.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2254 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 952ee3ce-fed9-41b6-a4da-92e0eed8b084 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5712025-01-15T02:30:34.825ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf70ddc4-7451-4d22-af9b-91cc2bf95b25 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5722025-01-15T02:30:34.825ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf70ddc4-7451-4d22-af9b-91cc2bf95b25 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5732025-01-15T02:30:34.827ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf70ddc4-7451-4d22-af9b-91cc2bf95b25 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5742025-01-15T02:30:35.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 482 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = f7013b32-f765-44fd-9a73-91e86a3223c1 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
5752025-01-15T02:30:35.847ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 940bb0a4-262f-4cbf-8ddd-12e51b474545 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5762025-01-15T02:30:35.848ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 940bb0a4-262f-4cbf-8ddd-12e51b474545 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5772025-01-15T02:30:35.849ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 940bb0a4-262f-4cbf-8ddd-12e51b474545 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5782025-01-15T02:30:36.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3ee817ba-2b20-45ff-be28-09f186e7972e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5792025-01-15T02:30:36.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3ee817ba-2b20-45ff-be28-09f186e7972e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5802025-01-15T02:30:36.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1995 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3ee817ba-2b20-45ff-be28-09f186e7972e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5812025-01-15T02:30:37.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a35d31c8-ab8c-4d01-9983-ebd91f195783 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5822025-01-15T02:30:37.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a35d31c8-ab8c-4d01-9983-ebd91f195783 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5832025-01-15T02:30:37.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2222 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a35d31c8-ab8c-4d01-9983-ebd91f195783 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5842025-01-15T02:30:38.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 85a887f1-6d22-44cf-82ff-efd63949f71e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5852025-01-15T02:30:38.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 85a887f1-6d22-44cf-82ff-efd63949f71e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5862025-01-15T02:30:38.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2619 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 85a887f1-6d22-44cf-82ff-efd63949f71e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5872025-01-15T02:30:40.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7b0dd4b4-d2d6-49b4-bf56-be2bd39f28eb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5882025-01-15T02:30:40.007ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7b0dd4b4-d2d6-49b4-bf56-be2bd39f28eb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5892025-01-15T02:30:40.009ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2652 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7b0dd4b4-d2d6-49b4-bf56-be2bd39f28eb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
5902025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
5912025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
5922025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5932025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
5942025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5952025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5962025-01-15T02:30:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5972025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
5982025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5992025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6002025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
6012025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
6022025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6032025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Timeout
6042025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6052025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6062025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
6072025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
6082025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6092025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6102025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
6112025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6122025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
6132025-01-15T02:30:40.507ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6142025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6152025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
6162025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6172025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
6182025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6192025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
6202025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6212025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6222025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6232025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6242025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6252025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6262025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6272025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6282025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6292025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6302025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6312025-01-15T02:30:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6322025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
6332025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
6342025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6352025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6362025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
6372025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6382025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
6392025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6402025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6412025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6422025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6432025-01-15T02:30:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6442025-01-15T02:30:40.510ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3527, tv_nsec: 344661385 } })
6452025-01-15T02:30:40.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.012168ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6462025-01-15T02:30:40.510ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6472025-01-15T02:30:40.511ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6482025-01-15T02:30:40.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6492025-01-15T02:30:40.513ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3527, tv_nsec: 348135009 } })
6502025-01-15T02:30:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6512025-01-15T02:30:40.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6522025-01-15T02:30:40.514ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6532025-01-15T02:30:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6542025-01-15T02:30:40.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6552025-01-15T02:30:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6562025-01-15T02:30:40.515ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6572025-01-15T02:30:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6582025-01-15T02:30:40.516ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6592025-01-15T02:30:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6602025-01-15T02:30:40.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6612025-01-15T02:30:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6622025-01-15T02:30:40.516ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6632025-01-15T02:30:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6642025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6652025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6662025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6672025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6682025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6702025-01-15T02:30:40.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6712025-01-15T02:30:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6722025-01-15T02:30:40.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6732025-01-15T02:30:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6742025-01-15T02:30:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6752025-01-15T02:30:40.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6762025-01-15T02:30:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6772025-01-15T02:30:40.526ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782025-01-15T02:30:40.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6792025-01-15T02:30:40.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6802025-01-15T02:30:40.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3527, tv_nsec: 361065368 } })
6812025-01-15T02:30:40.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
6822025-01-15T02:30:40.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6832025-01-15T02:30:40.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6842025-01-15T02:30:40.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6852025-01-15T02:30:40.528ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6862025-01-15T02:30:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6872025-01-15T02:30:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6882025-01-15T02:30:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6892025-01-15T02:30:40.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6902025-01-15T02:30:40.531ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6912025-01-15T02:30:40.531ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.854954ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6922025-01-15T02:30:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6932025-01-15T02:30:40.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6942025-01-15T02:30:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6952025-01-15T02:30:40.548ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6962025-01-15T02:30:40.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6972025-01-15T02:30:40.553ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 44.946713ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
6982025-01-15T02:30:40.555ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6992025-01-15T02:30:40.555ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7002025-01-15T02:30:40.555ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7012025-01-15T02:30:40.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7022025-01-15T02:30:40.555ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 47.651806ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
7032025-01-15T02:30:40.557ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7042025-01-15T02:30:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7052025-01-15T02:30:40.559ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7062025-01-15T02:30:40.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7072025-01-15T02:30:40.562ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7082025-01-15T02:30:40.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7092025-01-15T02:30:40.565ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7102025-01-15T02:30:40.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7112025-01-15T02:30:40.566ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7122025-01-15T02:30:40.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7132025-01-15T02:30:40.566ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7142025-01-15T02:30:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7152025-01-15T02:30:40.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162025-01-15T02:30:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
7172025-01-15T02:30:40.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7182025-01-15T02:30:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7192025-01-15T02:30:40.589ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7202025-01-15T02:30:40.589ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7212025-01-15T02:30:40.590ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e1c51fc-1bd9-4053-89c4-2d4cbc8ec15d", "content-length": "26", "date": "Wed, 15 Jan 2025 02:30:39 GMT"} })
7222025-01-15T02:30:40.590ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7232025-01-15T02:30:40.592ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7242025-01-15T02:30:40.595ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7252025-01-15T02:30:40.595ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7262025-01-15T02:30:40.595ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "753ae0c1-e5fb-4a58-871e-0a2eb66f162c", "content-length": "26", "date": "Wed, 15 Jan 2025 02:30:39 GMT"} })
7272025-01-15T02:30:40.595ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7282025-01-15T02:30:40.597ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7292025-01-15T02:30:40.627ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
7302025-01-15T02:30:40.627ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7312025-01-15T02:30:40.627ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e8f4656-70a0-43bb-8dee-43464c440ee4", "content-length": "26", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
7322025-01-15T02:30:40.627ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7332025-01-15T02:30:40.629ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
7342025-01-15T02:30:40.633ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7352025-01-15T02:30:40.633ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7362025-01-15T02:30:40.633ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "076c0eaa-142d-44a0-9f40-2cafd34319a5", "content-length": "372", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
7372025-01-15T02:30:40.633ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7382025-01-15T02:30:40.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7392025-01-15T02:30:40.635ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 128.812976ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7402025-01-15T02:30:40.637ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7412025-01-15T02:30:40.637ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7422025-01-15T02:30:40.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7432025-01-15T02:30:40.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7442025-01-15T02:30:40.639ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7452025-01-15T02:30:40.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7462025-01-15T02:30:40.654ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7472025-01-15T02:30:40.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
7482025-01-15T02:30:40.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7492025-01-15T02:30:40.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7502025-01-15T02:30:40.660ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 152.957338ms iteration = 8 status = 0
7512025-01-15T02:30:40.663ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
7522025-01-15T02:30:40.663ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 155.872926ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
7532025-01-15T02:30:40.675ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7542025-01-15T02:30:40.675ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7552025-01-15T02:30:40.675ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7562025-01-15T02:30:40.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7572025-01-15T02:30:40.676ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7582025-01-15T02:30:40.677ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 168.754233ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7592025-01-15T02:30:40.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7602025-01-15T02:30:40.678ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 171.520478ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7612025-01-15T02:30:40.695ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
7622025-01-15T02:30:40.695ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 188.371793ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7632025-01-15T02:30:40.695ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 188.383653ms iteration = 10 status = {"physical_disks_added":0}
7642025-01-15T02:30:40.696ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7652025-01-15T02:30:40.696ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
7662025-01-15T02:30:40.696ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7672025-01-15T02:30:40.696ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "435d5b5f-c1f7-4956-87ef-05fcb7410fe3", "content-length": "1", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
7682025-01-15T02:30:40.697ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9310a663-72b5-4ee4-beb0-3f041a17e354", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
7692025-01-15T02:30:40.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7702025-01-15T02:30:40.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7712025-01-15T02:30:40.698ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7722025-01-15T02:30:40.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7732025-01-15T02:30:40.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7742025-01-15T02:30:40.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7752025-01-15T02:30:40.700ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 194.478687ms iteration = 8 status = {}
7762025-01-15T02:30:40.701ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 195.115565ms iteration = 8 status = null
7772025-01-15T02:30:40.703ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7782025-01-15T02:30:40.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7792025-01-15T02:30:40.706ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7802025-01-15T02:30:40.706ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7812025-01-15T02:30:40.706ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "07f6fd3a-5d57-45e5-9587-0283ba5ef1d0", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
7822025-01-15T02:30:40.708ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 200.976924ms iteration = 8 status = {}
7832025-01-15T02:30:40.708ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7842025-01-15T02:30:40.708ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7852025-01-15T02:30:40.708ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 201.736854ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7862025-01-15T02:30:40.713ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 204.799016ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7872025-01-15T02:30:40.743ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7882025-01-15T02:30:40.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7892025-01-15T02:30:40.746ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7902025-01-15T02:30:40.747ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7912025-01-15T02:30:40.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7922025-01-15T02:30:40.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7932025-01-15T02:30:40.749ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942025-01-15T02:30:40.750ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7952025-01-15T02:30:40.750ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7962025-01-15T02:30:40.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7972025-01-15T02:30:40.776ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7982025-01-15T02:30:40.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7992025-01-15T02:30:40.779ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
8002025-01-15T02:30:40.779ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8012025-01-15T02:30:40.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8022025-01-15T02:30:40.790ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8032025-01-15T02:30:40.790ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "15f3e573-13ab-4af6-b279-3f9344b8903c", "content-length": "134", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8042025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8052025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
8062025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8072025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8082025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8092025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8102025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8112025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8122025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8132025-01-15T02:30:40.790ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8142025-01-15T02:30:40.791ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
8152025-01-15T02:30:40.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8162025-01-15T02:30:40.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8172025-01-15T02:30:40.793ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8182025-01-15T02:30:40.793ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6881f4e4-1d5c-459a-a42e-e94892eb4821", "content-length": "2", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8192025-01-15T02:30:40.793ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8202025-01-15T02:30:40.793ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "aef724bd-fa3c-4b47-9615-95d3f11378be", "content-length": "52", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8212025-01-15T02:30:40.793ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
8222025-01-15T02:30:40.794ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8232025-01-15T02:30:40.836ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8242025-01-15T02:30:40.869ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f0b1c48-d982-41d8-9dbe-c2867b882cc0", "content-length": "691", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8252025-01-15T02:30:40.869ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8262025-01-15T02:30:40.869ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8272025-01-15T02:30:40.869ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8282025-01-15T02:30:40.870ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 363.53993ms iteration = 8 status = {}
8292025-01-15T02:30:40.912ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
8302025-01-15T02:30:40.912ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
8312025-01-15T02:30:40.944ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
8322025-01-15T02:30:40.944ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:38912
8332025-01-15T02:30:40.944ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:38912 req_id = 5b7e378c-c5d2-43ee-8d6c-aa865b1b9a00 roles = RoleSet { roles: {} } uri = /metrics/producers
8342025-01-15T02:30:40.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:38912 req_id = 5b7e378c-c5d2-43ee-8d6c-aa865b1b9a00 resource = Database result = Ok(()) uri = /metrics/producers
8352025-01-15T02:30:40.960ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "27cbe932-2bec-4375-a51b-ae7b4c8bd093", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8362025-01-15T02:30:40.960ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
8372025-01-15T02:30:40.960ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8382025-01-15T02:30:40.961ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a19a7371-7a65-41fb-b8c7-4214e7e97eee", "content-length": "682", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8392025-01-15T02:30:40.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8402025-01-15T02:30:40.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8412025-01-15T02:30:40.964ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422025-01-15T02:30:40.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8432025-01-15T02:30:40.967ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8442025-01-15T02:30:40.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8452025-01-15T02:30:40.969ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8462025-01-15T02:30:40.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8472025-01-15T02:30:40.977ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 470.192178ms iteration = 8 status = {}
8482025-01-15T02:30:40.985ZINFOnexus: assigned collector to new producer collector_id = f71d33aa-7320-48b9-81d2-899a100e78cd file = nexus/src/app/oximeter.rs:76 producer_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
8492025-01-15T02:30:40.985ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40876 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:38912 req_id = 5b7e378c-c5d2-43ee-8d6c-aa865b1b9a00 response_code = 201 uri = /metrics/producers
8502025-01-15T02:30:40.985ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "5b7e378c-c5d2-43ee-8d6c-aa865b1b9a00", "content-length": "41", "date": "Wed, 15 Jan 2025 02:30:40 GMT"} })
8512025-01-15T02:30:40.986ZDEBGnexus (producer-registration-task): registered with nexus successfully
8522025-01-15T02:30:40.986ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
8532025-01-15T02:30:41.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ac1bed44-06b8-463e-b608-e1722396f1d1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8542025-01-15T02:30:41.030ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ac1bed44-06b8-463e-b608-e1722396f1d1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8552025-01-15T02:30:41.031ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2183 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ac1bed44-06b8-463e-b608-e1722396f1d1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8562025-01-15T02:30:42.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a7c0783-96f8-4393-bae0-4898744cc8e8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8572025-01-15T02:30:42.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a7c0783-96f8-4393-bae0-4898744cc8e8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8582025-01-15T02:30:42.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1922 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a7c0783-96f8-4393-bae0-4898744cc8e8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8592025-01-15T02:30:43.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a3464c2-8bf3-4369-9b1c-ef2e71899923 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8602025-01-15T02:30:43.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a3464c2-8bf3-4369-9b1c-ef2e71899923 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8612025-01-15T02:30:43.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a3464c2-8bf3-4369-9b1c-ef2e71899923 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8622025-01-15T02:30:44.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 99b183cd-b158-4ecd-9073-98737d425bc7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8632025-01-15T02:30:44.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 99b183cd-b158-4ecd-9073-98737d425bc7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8642025-01-15T02:30:44.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2582 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 99b183cd-b158-4ecd-9073-98737d425bc7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8652025-01-15T02:30:45.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a5a0970-2f67-47d1-8b57-413a2aa112bc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8662025-01-15T02:30:45.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a5a0970-2f67-47d1-8b57-413a2aa112bc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8672025-01-15T02:30:45.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2383 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a5a0970-2f67-47d1-8b57-413a2aa112bc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8682025-01-15T02:30:45.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 425 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = b500772d-5067-4f02-b585-ae27367d09ab response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
8692025-01-15T02:30:46.142ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c0e9a97-d524-46f5-a096-93adbceeed67 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8702025-01-15T02:30:46.142ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c0e9a97-d524-46f5-a096-93adbceeed67 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8712025-01-15T02:30:46.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2065 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c0e9a97-d524-46f5-a096-93adbceeed67 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8722025-01-15T02:30:47.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b27337cb-a543-4f7a-985d-ec5a9343f67e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8732025-01-15T02:30:47.165ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b27337cb-a543-4f7a-985d-ec5a9343f67e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8742025-01-15T02:30:47.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1961 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b27337cb-a543-4f7a-985d-ec5a9343f67e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8752025-01-15T02:30:48.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb028a49-54d5-4db8-9737-ef41b274f118 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8762025-01-15T02:30:48.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb028a49-54d5-4db8-9737-ef41b274f118 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8772025-01-15T02:30:48.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1874 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb028a49-54d5-4db8-9737-ef41b274f118 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8782025-01-15T02:30:49.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 697c910d-cdbc-45ee-9a39-d5564a6efc8b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8792025-01-15T02:30:49.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 697c910d-cdbc-45ee-9a39-d5564a6efc8b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8802025-01-15T02:30:49.210ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2426 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 697c910d-cdbc-45ee-9a39-d5564a6efc8b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8812025-01-15T02:30:50.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c4dfe18-5266-47a1-a494-c40954143488 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8822025-01-15T02:30:50.230ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c4dfe18-5266-47a1-a494-c40954143488 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8832025-01-15T02:30:50.231ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2054 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c4dfe18-5266-47a1-a494-c40954143488 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
8842025-01-15T02:30:50.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
8852025-01-15T02:30:50.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
8862025-01-15T02:30:50.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8872025-01-15T02:30:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8882025-01-15T02:30:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8892025-01-15T02:30:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8902025-01-15T02:30:50.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8912025-01-15T02:30:50.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8922025-01-15T02:30:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8932025-01-15T02:30:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8942025-01-15T02:30:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8952025-01-15T02:30:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8962025-01-15T02:30:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8972025-01-15T02:30:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8982025-01-15T02:30:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8992025-01-15T02:30:50.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
9002025-01-15T02:30:50.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9012025-01-15T02:30:50.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9022025-01-15T02:30:50.529ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
9032025-01-15T02:30:50.529ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.237573ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
9042025-01-15T02:30:50.537ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.896987ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9052025-01-15T02:30:51.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e91391b-2709-410b-9fcc-cd83b09d17a3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9062025-01-15T02:30:51.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e91391b-2709-410b-9fcc-cd83b09d17a3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9072025-01-15T02:30:51.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1782 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e91391b-2709-410b-9fcc-cd83b09d17a3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9082025-01-15T02:30:52.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9181ef4c-833e-43f4-b7f3-049decace751 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9092025-01-15T02:30:52.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9181ef4c-833e-43f4-b7f3-049decace751 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9102025-01-15T02:30:52.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2401 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9181ef4c-833e-43f4-b7f3-049decace751 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9112025-01-15T02:30:53.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d13091cf-732b-45d8-869c-3efa687fcd87 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9122025-01-15T02:30:53.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d13091cf-732b-45d8-869c-3efa687fcd87 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9132025-01-15T02:30:53.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2133 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d13091cf-732b-45d8-869c-3efa687fcd87 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9142025-01-15T02:30:54.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82108d6e-81e9-4151-baaf-547d48efa436 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9152025-01-15T02:30:54.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82108d6e-81e9-4151-baaf-547d48efa436 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9162025-01-15T02:30:54.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2294 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82108d6e-81e9-4151-baaf-547d48efa436 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9172025-01-15T02:30:55.336ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 18fe2596-1137-4463-b048-d17476ce8817 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9182025-01-15T02:30:55.336ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 18fe2596-1137-4463-b048-d17476ce8817 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9192025-01-15T02:30:55.338ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2515 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 18fe2596-1137-4463-b048-d17476ce8817 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9202025-01-15T02:30:55.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 446 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = a1b6a47d-5767-490c-97c0-536f0d3243ec response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
9212025-01-15T02:30:56.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23c96936-3b62-49ea-a988-e1935937839b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9222025-01-15T02:30:56.358ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23c96936-3b62-49ea-a988-e1935937839b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9232025-01-15T02:30:56.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23c96936-3b62-49ea-a988-e1935937839b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9242025-01-15T02:30:57.379ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f0b15a8b-0239-4cc0-b633-4dc4752a3df6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9252025-01-15T02:30:57.379ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f0b15a8b-0239-4cc0-b633-4dc4752a3df6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9262025-01-15T02:30:57.381ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1815 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f0b15a8b-0239-4cc0-b633-4dc4752a3df6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9272025-01-15T02:30:58.399ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2eea9c27-5529-45b9-8de6-548aabc1b2a7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9282025-01-15T02:30:58.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2eea9c27-5529-45b9-8de6-548aabc1b2a7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9292025-01-15T02:30:58.402ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2354 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2eea9c27-5529-45b9-8de6-548aabc1b2a7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9302025-01-15T02:30:59.422ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 236ebb9b-56a0-4ac3-8c86-7d963f5dff64 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9312025-01-15T02:30:59.422ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 236ebb9b-56a0-4ac3-8c86-7d963f5dff64 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9322025-01-15T02:30:59.424ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 236ebb9b-56a0-4ac3-8c86-7d963f5dff64 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9332025-01-15T02:31:00.443ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6585bd5a-6c49-406b-b8e7-420cdf942992 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9342025-01-15T02:31:00.443ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6585bd5a-6c49-406b-b8e7-420cdf942992 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9352025-01-15T02:31:00.445ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2368 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6585bd5a-6c49-406b-b8e7-420cdf942992 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9362025-01-15T02:31:00.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
9372025-01-15T02:31:00.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
9382025-01-15T02:31:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9392025-01-15T02:31:00.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9402025-01-15T02:31:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9412025-01-15T02:31:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9422025-01-15T02:31:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9432025-01-15T02:31:00.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9442025-01-15T02:31:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9452025-01-15T02:31:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9462025-01-15T02:31:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9472025-01-15T02:31:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9482025-01-15T02:31:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9492025-01-15T02:31:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9502025-01-15T02:31:00.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9512025-01-15T02:31:00.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
9522025-01-15T02:31:00.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9532025-01-15T02:31:00.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9542025-01-15T02:31:00.528ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
9552025-01-15T02:31:00.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.179901ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
9562025-01-15T02:31:00.538ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.13613ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9572025-01-15T02:31:01.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c72b492-dfa5-434e-80df-876486bd13be roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9582025-01-15T02:31:01.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c72b492-dfa5-434e-80df-876486bd13be resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9592025-01-15T02:31:01.467ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2270 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c72b492-dfa5-434e-80df-876486bd13be response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9602025-01-15T02:31:02.488ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9f053cf4-20a1-4ff7-ab87-b84690bd87dd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9612025-01-15T02:31:02.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9f053cf4-20a1-4ff7-ab87-b84690bd87dd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9622025-01-15T02:31:02.490ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1892 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9f053cf4-20a1-4ff7-ab87-b84690bd87dd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9632025-01-15T02:31:03.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 57c2c81f-144b-4407-ae87-0d6b5eb09fbb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9642025-01-15T02:31:03.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 57c2c81f-144b-4407-ae87-0d6b5eb09fbb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9652025-01-15T02:31:03.510ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2143 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 57c2c81f-144b-4407-ae87-0d6b5eb09fbb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9662025-01-15T02:31:04.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 833e5c49-fcb6-4fa3-a87f-5885e6b67479 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9672025-01-15T02:31:04.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 833e5c49-fcb6-4fa3-a87f-5885e6b67479 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9682025-01-15T02:31:04.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2454 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 833e5c49-fcb6-4fa3-a87f-5885e6b67479 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9692025-01-15T02:31:05.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ab682a8b-451b-4e04-a1b9-1ec1e8ef8ec9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9702025-01-15T02:31:05.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ab682a8b-451b-4e04-a1b9-1ec1e8ef8ec9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9712025-01-15T02:31:05.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1977 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ab682a8b-451b-4e04-a1b9-1ec1e8ef8ec9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9722025-01-15T02:31:05.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 504 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 3d59f59b-8dce-4330-b2c1-a5ccbbf770b8 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
9732025-01-15T02:31:06.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7ea5a496-4da5-4550-9a21-5e4c9a63d58b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9742025-01-15T02:31:06.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7ea5a496-4da5-4550-9a21-5e4c9a63d58b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9752025-01-15T02:31:06.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2572 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7ea5a496-4da5-4550-9a21-5e4c9a63d58b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9762025-01-15T02:31:07.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6194dd82-0cd4-468a-97a1-1ef77ef25a8f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9772025-01-15T02:31:07.598ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6194dd82-0cd4-468a-97a1-1ef77ef25a8f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9782025-01-15T02:31:07.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2490 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6194dd82-0cd4-468a-97a1-1ef77ef25a8f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9792025-01-15T02:31:08.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cfd28766-e756-42b2-833d-c875961f182c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9802025-01-15T02:31:08.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cfd28766-e756-42b2-833d-c875961f182c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9812025-01-15T02:31:08.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2101 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cfd28766-e756-42b2-833d-c875961f182c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9822025-01-15T02:31:09.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 87ffb5cc-1e53-478a-b1db-a6a1df629de6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9832025-01-15T02:31:09.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 87ffb5cc-1e53-478a-b1db-a6a1df629de6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9842025-01-15T02:31:09.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11043 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 87ffb5cc-1e53-478a-b1db-a6a1df629de6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
9852025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
9862025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
9872025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
9882025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
9892025-01-15T02:31:10.506ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:21:10.506493471 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9902025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
9912025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
9922025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9932025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9942025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9952025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
9962025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9972025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9982025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
9992025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
10002025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
10012025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10022025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10032025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
10042025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10052025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10062025-01-15T02:31:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10072025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10082025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10092025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10102025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
10112025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
10122025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
10132025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10142025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10152025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10162025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10172025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
10182025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
10192025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
10202025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10212025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10222025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10232025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
10242025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10252025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
10262025-01-15T02:31:10.507ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10272025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
10282025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10292025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
10302025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10312025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10322025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10332025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
10342025-01-15T02:31:10.507ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
10352025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.901\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
10362025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
10372025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10382025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
10392025-01-15T02:31:10.507ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
10402025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10412025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
10422025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10432025-01-15T02:31:10.507ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3557, tv_nsec: 341941142 } })
10442025-01-15T02:31:10.507ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3557, tv_nsec: 341934791 } })
10452025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10462025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10472025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10482025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10492025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10502025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10512025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10522025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10532025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10542025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10552025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10562025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10572025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10582025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
10592025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10602025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
10612025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10622025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
10632025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10642025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
10652025-01-15T02:31:10.507ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10662025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
10672025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10682025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10692025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
10702025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.47088ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
10712025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10722025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
10732025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.544221ms iteration = 5 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
10742025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10752025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10762025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10772025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
10782025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10792025-01-15T02:31:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10802025-01-15T02:31:10.509ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10812025-01-15T02:31:10.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10822025-01-15T02:31:10.510ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:21:10.506493471 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10832025-01-15T02:31:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.151132ms iteration = 5 status = {"expiration":"2025-01-15T02:21:10.506493471Z","pruned":[]}
10842025-01-15T02:31:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.212364ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10852025-01-15T02:31:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10862025-01-15T02:31:10.511ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10872025-01-15T02:31:10.511ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10882025-01-15T02:31:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10892025-01-15T02:31:10.511ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3557, tv_nsec: 346316110 } })
10902025-01-15T02:31:10.511ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3557, tv_nsec: 346458694 } })
10912025-01-15T02:31:10.511ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
10922025-01-15T02:31:10.512ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
10932025-01-15T02:31:10.512ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10942025-01-15T02:31:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10952025-01-15T02:31:10.515ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10962025-01-15T02:31:10.515ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10972025-01-15T02:31:10.515ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10982025-01-15T02:31:10.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10992025-01-15T02:31:10.516ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11002025-01-15T02:31:10.516ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
11012025-01-15T02:31:10.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11022025-01-15T02:31:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11032025-01-15T02:31:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11042025-01-15T02:31:10.518ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11052025-01-15T02:31:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11062025-01-15T02:31:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11072025-01-15T02:31:10.518ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11082025-01-15T02:31:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11092025-01-15T02:31:10.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11102025-01-15T02:31:10.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11112025-01-15T02:31:10.521ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11122025-01-15T02:31:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11132025-01-15T02:31:10.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11142025-01-15T02:31:10.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11152025-01-15T02:31:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11162025-01-15T02:31:10.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11172025-01-15T02:31:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11182025-01-15T02:31:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11192025-01-15T02:31:10.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11202025-01-15T02:31:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11212025-01-15T02:31:10.524ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11222025-01-15T02:31:10.524ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11232025-01-15T02:31:10.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11242025-01-15T02:31:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11252025-01-15T02:31:10.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11262025-01-15T02:31:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11272025-01-15T02:31:10.525ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11282025-01-15T02:31:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
11292025-01-15T02:31:10.527ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
11302025-01-15T02:31:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
11312025-01-15T02:31:10.528ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.663477ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
11322025-01-15T02:31:10.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11332025-01-15T02:31:10.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11342025-01-15T02:31:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11352025-01-15T02:31:10.537ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
11362025-01-15T02:31:10.537ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 30.177855ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
11372025-01-15T02:31:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11382025-01-15T02:31:10.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11392025-01-15T02:31:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11402025-01-15T02:31:10.541ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.47764ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11412025-01-15T02:31:10.541ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3557, tv_nsec: 376560036 } })
11422025-01-15T02:31:10.542ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
11432025-01-15T02:31:10.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11442025-01-15T02:31:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11452025-01-15T02:31:10.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11462025-01-15T02:31:10.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11472025-01-15T02:31:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11482025-01-15T02:31:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11492025-01-15T02:31:10.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11502025-01-15T02:31:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
11512025-01-15T02:31:10.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11522025-01-15T02:31:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11532025-01-15T02:31:10.567ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11542025-01-15T02:31:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11552025-01-15T02:31:10.570ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11562025-01-15T02:31:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11572025-01-15T02:31:10.577ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11582025-01-15T02:31:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11592025-01-15T02:31:10.580ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11602025-01-15T02:31:10.580ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.465775ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
11612025-01-15T02:31:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11622025-01-15T02:31:10.582ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 74.343386ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11632025-01-15T02:31:10.585ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11642025-01-15T02:31:10.585ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11652025-01-15T02:31:10.585ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11662025-01-15T02:31:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11672025-01-15T02:31:10.587ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
11682025-01-15T02:31:10.587ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
11692025-01-15T02:31:10.587ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 79.092715ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
11702025-01-15T02:31:10.588ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
11712025-01-15T02:31:10.588ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
11722025-01-15T02:31:10.588ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
11732025-01-15T02:31:10.588ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11742025-01-15T02:31:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11752025-01-15T02:31:10.589ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
11762025-01-15T02:31:10.589ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 81.958372ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
11772025-01-15T02:31:10.589ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11782025-01-15T02:31:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11792025-01-15T02:31:10.593ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11802025-01-15T02:31:10.594ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 86.778312ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
11812025-01-15T02:31:10.594ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
11822025-01-15T02:31:10.594ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
11832025-01-15T02:31:10.594ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
11842025-01-15T02:31:10.594ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11852025-01-15T02:31:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11862025-01-15T02:31:10.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11872025-01-15T02:31:10.596ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11882025-01-15T02:31:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11892025-01-15T02:31:10.598ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11902025-01-15T02:31:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11912025-01-15T02:31:10.600ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
11922025-01-15T02:31:10.600ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 93.644867ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
11932025-01-15T02:31:10.600ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11942025-01-15T02:31:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11952025-01-15T02:31:10.601ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11962025-01-15T02:31:10.602ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972025-01-15T02:31:10.604ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
11982025-01-15T02:31:10.604ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
11992025-01-15T02:31:10.604ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 96.933756ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
12002025-01-15T02:31:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12012025-01-15T02:31:10.604ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12022025-01-15T02:31:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12032025-01-15T02:31:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12042025-01-15T02:31:10.606ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12052025-01-15T02:31:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12062025-01-15T02:31:10.621ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12072025-01-15T02:31:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12082025-01-15T02:31:10.629ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12092025-01-15T02:31:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12102025-01-15T02:31:10.630ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12112025-01-15T02:31:10.630ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12122025-01-15T02:31:10.630ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 123.584565ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12132025-01-15T02:31:10.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12142025-01-15T02:31:10.636ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 129.118914ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12152025-01-15T02:31:10.638ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12162025-01-15T02:31:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12172025-01-15T02:31:10.639ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12182025-01-15T02:31:10.639ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12192025-01-15T02:31:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12202025-01-15T02:31:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12212025-01-15T02:31:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12222025-01-15T02:31:10.641ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12232025-01-15T02:31:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12242025-01-15T02:31:10.645ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12252025-01-15T02:31:10.645ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12262025-01-15T02:31:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
12272025-01-15T02:31:10.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12282025-01-15T02:31:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12292025-01-15T02:31:10.650ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
12302025-01-15T02:31:10.650ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12312025-01-15T02:31:10.651ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a489ed3c-ab81-4f51-9719-eec9a0b7eb1c", "content-length": "26", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
12322025-01-15T02:31:10.651ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12332025-01-15T02:31:10.653ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12342025-01-15T02:31:10.657ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12352025-01-15T02:31:10.664ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
12362025-01-15T02:31:10.664ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12372025-01-15T02:31:10.664ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12382025-01-15T02:31:10.664ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dd85483-b318-458b-8ca5-5951fc46eb76", "content-length": "26", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
12392025-01-15T02:31:10.664ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12402025-01-15T02:31:10.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12412025-01-15T02:31:10.666ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12422025-01-15T02:31:10.681ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 76cf4462-36ba-4295-9af8-5b5bf6542cee roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
12432025-01-15T02:31:10.682ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 76cf4462-36ba-4295-9af8-5b5bf6542cee resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
12442025-01-15T02:31:10.706ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
12452025-01-15T02:31:10.706ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12462025-01-15T02:31:10.707ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "26f8ec71-6d82-4cc9-b314-c0bff2ef784e", "content-length": "26", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
12472025-01-15T02:31:10.707ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12482025-01-15T02:31:10.709ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
12492025-01-15T02:31:10.711ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 204.032536ms iteration = 11 status = {"physical_disks_added":0}
12502025-01-15T02:31:10.743ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
12512025-01-15T02:31:10.743ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12522025-01-15T02:31:10.744ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b9df8c9-b44f-4063-a160-cc4af839e7ff", "content-length": "372", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
12532025-01-15T02:31:10.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12542025-01-15T02:31:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12552025-01-15T02:31:10.745ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12562025-01-15T02:31:10.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12572025-01-15T02:31:10.747ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12582025-01-15T02:31:10.747ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12592025-01-15T02:31:10.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12602025-01-15T02:31:10.748ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 241.490747ms iteration = 9 status = 0
12612025-01-15T02:31:10.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12622025-01-15T02:31:10.749ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12632025-01-15T02:31:10.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12642025-01-15T02:31:10.753ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71252 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 76cf4462-36ba-4295-9af8-5b5bf6542cee response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
12652025-01-15T02:31:10.754ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
12662025-01-15T02:31:10.754ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 246.909133ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
12672025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12682025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
12692025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12702025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12712025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12722025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12732025-01-15T02:31:10.777ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12742025-01-15T02:31:10.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12752025-01-15T02:31:10.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12762025-01-15T02:31:10.780ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12772025-01-15T02:31:10.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12782025-01-15T02:31:10.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12792025-01-15T02:31:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12802025-01-15T02:31:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12812025-01-15T02:31:10.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12822025-01-15T02:31:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12832025-01-15T02:31:10.784ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
12842025-01-15T02:31:10.784ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
12852025-01-15T02:31:10.784ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
12862025-01-15T02:31:10.784ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 277.815087ms iteration = 5 status = {"generation":2}
12872025-01-15T02:31:10.785ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
12882025-01-15T02:31:10.785ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
12892025-01-15T02:31:10.785ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
12902025-01-15T02:31:10.785ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 278.886145ms iteration = 5 status = {"generation":1}
12912025-01-15T02:31:10.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 278.780153ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
12922025-01-15T02:31:10.786ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12932025-01-15T02:31:10.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12942025-01-15T02:31:10.790ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
12952025-01-15T02:31:10.791ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12962025-01-15T02:31:10.791ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a88d6d88-529c-4642-9c8b-fe78d4745349", "content-length": "1", "date": "Wed, 15 Jan 2025 02:31:09 GMT"} })
12972025-01-15T02:31:10.792ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12982025-01-15T02:31:10.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12992025-01-15T02:31:10.793ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
13002025-01-15T02:31:10.793ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13012025-01-15T02:31:10.793ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c4fb5824-6a12-4151-bffe-d8b66c348a10", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13022025-01-15T02:31:10.793ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13032025-01-15T02:31:10.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13042025-01-15T02:31:10.794ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 288.077603ms iteration = 9 status = null
13052025-01-15T02:31:10.798ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
13062025-01-15T02:31:10.798ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 291.889217ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
13072025-01-15T02:31:10.800ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13082025-01-15T02:31:10.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13092025-01-15T02:31:10.801ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13102025-01-15T02:31:10.801ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "86f2dc66-b2be-458b-b104-9448212e6a40", "date": "Wed, 15 Jan 2025 02:31:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13112025-01-15T02:31:10.801ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13122025-01-15T02:31:10.803ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13132025-01-15T02:31:10.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13142025-01-15T02:31:10.803ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "aaeb9d0d-9bb6-4d3f-ba49-02d2f5285e57", "date": "Wed, 15 Jan 2025 02:31:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13152025-01-15T02:31:10.803ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13162025-01-15T02:31:10.805ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13172025-01-15T02:31:10.805ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 299.310376ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
13182025-01-15T02:31:10.806ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13192025-01-15T02:31:10.806ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13202025-01-15T02:31:10.806ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d6c4faa3-039f-445a-9cc5-e2a917c3884e", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13212025-01-15T02:31:10.808ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 301.380573ms iteration = 9 status = {}
13222025-01-15T02:31:10.809ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 303.111539ms iteration = 9 status = {}
13232025-01-15T02:31:10.816ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13242025-01-15T02:31:10.817ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13252025-01-15T02:31:10.817ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13262025-01-15T02:31:10.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13272025-01-15T02:31:10.844ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13282025-01-15T02:31:10.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13292025-01-15T02:31:10.845ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13302025-01-15T02:31:10.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13312025-01-15T02:31:10.860ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13322025-01-15T02:31:10.860ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "cb70d5f5-0afb-4bfa-822e-5a33a7250b9b", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13332025-01-15T02:31:10.860ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13342025-01-15T02:31:10.863ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1e9bae98-d02e-4fff-9429-e8f1e95dbc8a", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13352025-01-15T02:31:10.863ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13362025-01-15T02:31:10.864ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "aee6f3dd-b5fd-472f-998d-0da798aecd00", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13372025-01-15T02:31:10.864ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13382025-01-15T02:31:10.866ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13392025-01-15T02:31:10.866ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 359.581413ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13402025-01-15T02:31:10.904ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13412025-01-15T02:31:10.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13422025-01-15T02:31:10.906ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
13432025-01-15T02:31:10.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13442025-01-15T02:31:10.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13452025-01-15T02:31:10.915ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13462025-01-15T02:31:10.915ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "7acc34c5-f614-4e6f-8b99-2ff249a56a75", "content-length": "134", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13472025-01-15T02:31:10.915ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
13482025-01-15T02:31:10.915ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
13492025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
13502025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13512025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13522025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13532025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13542025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13552025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13562025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13572025-01-15T02:31:10.916ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
13582025-01-15T02:31:10.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13592025-01-15T02:31:10.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13602025-01-15T02:31:10.938ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13612025-01-15T02:31:10.938ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "22aeaa7d-86d3-4cf2-bb34-81f7f7ba2912", "content-length": "691", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13622025-01-15T02:31:10.938ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
13632025-01-15T02:31:10.938ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
13642025-01-15T02:31:10.938ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
13652025-01-15T02:31:10.939ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 432.009827ms iteration = 9 status = {}
13662025-01-15T02:31:10.940ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13672025-01-15T02:31:10.940ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "173a2405-2955-4259-81f2-6a04f2e57ef0", "content-length": "2", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13682025-01-15T02:31:10.940ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13692025-01-15T02:31:10.940ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d20ac1e-f879-4797-9f1c-994701cd152d", "content-length": "52", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13702025-01-15T02:31:10.940ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
13712025-01-15T02:31:10.940ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13722025-01-15T02:31:11.038ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3b002c93-981d-4781-b65c-f11c2f9ed03e", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13732025-01-15T02:31:11.038ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
13742025-01-15T02:31:11.038ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13752025-01-15T02:31:11.042ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "317b72d8-3576-42ef-9448-cc1b03b4494d", "content-length": "682", "date": "Wed, 15 Jan 2025 02:31:10 GMT"} })
13762025-01-15T02:31:11.042ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13772025-01-15T02:31:11.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13782025-01-15T02:31:11.046ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13792025-01-15T02:31:11.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13802025-01-15T02:31:11.049ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13812025-01-15T02:31:11.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13822025-01-15T02:31:11.051ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13832025-01-15T02:31:11.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13842025-01-15T02:31:11.058ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 551.621865ms iteration = 9 status = {}
13852025-01-15T02:31:11.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 69a4baa7-e219-4186-835a-55b492ea993a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13862025-01-15T02:31:11.774ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 69a4baa7-e219-4186-835a-55b492ea993a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13872025-01-15T02:31:11.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2428 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 69a4baa7-e219-4186-835a-55b492ea993a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13882025-01-15T02:31:12.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5a7d0f4-c86f-4d71-9588-eb7ef450b614 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13892025-01-15T02:31:12.796ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5a7d0f4-c86f-4d71-9588-eb7ef450b614 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13902025-01-15T02:31:12.798ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2495 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5a7d0f4-c86f-4d71-9588-eb7ef450b614 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13912025-01-15T02:31:13.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a470b678-c525-4151-aa28-57255e8cda3f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13922025-01-15T02:31:13.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a470b678-c525-4151-aa28-57255e8cda3f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13932025-01-15T02:31:13.820ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2477 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a470b678-c525-4151-aa28-57255e8cda3f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13942025-01-15T02:31:14.840ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 22744c3a-c2e4-40ca-82a3-5971be1af4ea roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13952025-01-15T02:31:14.841ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 22744c3a-c2e4-40ca-82a3-5971be1af4ea resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13962025-01-15T02:31:14.843ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 22744c3a-c2e4-40ca-82a3-5971be1af4ea response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13972025-01-15T02:31:15.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 482 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = d7b096eb-25a4-4e10-aeec-f5eb9de123fb response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
13982025-01-15T02:31:15.862ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe696a64-3f1f-4a48-8b49-e8989d5a5bb4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
13992025-01-15T02:31:15.863ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe696a64-3f1f-4a48-8b49-e8989d5a5bb4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14002025-01-15T02:31:15.865ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2551 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe696a64-3f1f-4a48-8b49-e8989d5a5bb4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14012025-01-15T02:31:16.883ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 74ddca0d-4ffd-4bff-927c-1c9f5227cf8a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14022025-01-15T02:31:16.884ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 74ddca0d-4ffd-4bff-927c-1c9f5227cf8a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14032025-01-15T02:31:16.885ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1980 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 74ddca0d-4ffd-4bff-927c-1c9f5227cf8a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14042025-01-15T02:31:17.905ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f3d65586-f9ad-4603-9415-fbac7f480204 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14052025-01-15T02:31:17.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f3d65586-f9ad-4603-9415-fbac7f480204 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14062025-01-15T02:31:17.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2747 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f3d65586-f9ad-4603-9415-fbac7f480204 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14072025-01-15T02:31:18.926ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 26da0804-b234-437c-93d1-ac6da69fb581 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14082025-01-15T02:31:18.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 26da0804-b234-437c-93d1-ac6da69fb581 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14092025-01-15T02:31:18.928ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2066 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 26da0804-b234-437c-93d1-ac6da69fb581 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14102025-01-15T02:31:19.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0497398a-add4-41af-907d-c30af37ecdef roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14112025-01-15T02:31:19.949ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0497398a-add4-41af-907d-c30af37ecdef resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14122025-01-15T02:31:19.951ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2034 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0497398a-add4-41af-907d-c30af37ecdef response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14132025-01-15T02:31:20.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
14142025-01-15T02:31:20.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
14152025-01-15T02:31:20.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14162025-01-15T02:31:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14172025-01-15T02:31:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14182025-01-15T02:31:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14192025-01-15T02:31:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14202025-01-15T02:31:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14212025-01-15T02:31:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14222025-01-15T02:31:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14232025-01-15T02:31:20.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14242025-01-15T02:31:20.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14252025-01-15T02:31:20.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14262025-01-15T02:31:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14272025-01-15T02:31:20.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14282025-01-15T02:31:20.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
14292025-01-15T02:31:20.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14302025-01-15T02:31:20.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14312025-01-15T02:31:20.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
14322025-01-15T02:31:20.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.283368ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
14332025-01-15T02:31:20.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.727242ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14342025-01-15T02:31:20.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 02a9b8c5-d32b-47ce-a8f5-99b30538df2b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14352025-01-15T02:31:20.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 02a9b8c5-d32b-47ce-a8f5-99b30538df2b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14362025-01-15T02:31:20.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1915 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 02a9b8c5-d32b-47ce-a8f5-99b30538df2b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14372025-01-15T02:31:21.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 427aad84-6f0f-4ba6-aca9-d91192ab4d81 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14382025-01-15T02:31:21.993ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 427aad84-6f0f-4ba6-aca9-d91192ab4d81 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14392025-01-15T02:31:21.995ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2656 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 427aad84-6f0f-4ba6-aca9-d91192ab4d81 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14402025-01-15T02:31:23.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 218eab34-4540-4922-901a-98e1cacba115 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14412025-01-15T02:31:23.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 218eab34-4540-4922-901a-98e1cacba115 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14422025-01-15T02:31:23.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2221 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 218eab34-4540-4922-901a-98e1cacba115 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14432025-01-15T02:31:24.037ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89a5ad55-8d27-40e6-8dd0-35b5ab309f96 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14442025-01-15T02:31:24.038ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89a5ad55-8d27-40e6-8dd0-35b5ab309f96 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14452025-01-15T02:31:24.039ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1654 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89a5ad55-8d27-40e6-8dd0-35b5ab309f96 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14462025-01-15T02:31:25.059ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 30e62883-1043-4acc-a6bb-70f65f655e19 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14472025-01-15T02:31:25.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 30e62883-1043-4acc-a6bb-70f65f655e19 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14482025-01-15T02:31:25.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 30e62883-1043-4acc-a6bb-70f65f655e19 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14492025-01-15T02:31:25.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 578 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = f7855980-885d-43cc-abeb-b2bef9f81b87 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
14502025-01-15T02:31:26.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5a4d1d5d-b15a-412d-b21e-744b2c37f98a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14512025-01-15T02:31:26.082ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5a4d1d5d-b15a-412d-b21e-744b2c37f98a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14522025-01-15T02:31:26.084ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2158 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5a4d1d5d-b15a-412d-b21e-744b2c37f98a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14532025-01-15T02:31:27.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5858022f-2bb8-4ab2-9b43-4145d3de4169 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14542025-01-15T02:31:27.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5858022f-2bb8-4ab2-9b43-4145d3de4169 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14552025-01-15T02:31:27.107ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2483 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5858022f-2bb8-4ab2-9b43-4145d3de4169 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14562025-01-15T02:31:28.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 204f8046-bfd5-4c9c-bdaf-99b4c030e5ef roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14572025-01-15T02:31:28.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 204f8046-bfd5-4c9c-bdaf-99b4c030e5ef resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14582025-01-15T02:31:28.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1890 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 204f8046-bfd5-4c9c-bdaf-99b4c030e5ef response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14592025-01-15T02:31:29.149ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc23200f-f767-4bf4-8a0b-290b6bfd5076 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14602025-01-15T02:31:29.149ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc23200f-f767-4bf4-8a0b-290b6bfd5076 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14612025-01-15T02:31:29.151ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1978 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc23200f-f767-4bf4-8a0b-290b6bfd5076 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14622025-01-15T02:31:30.171ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ca6670c-b2e7-4395-95b0-70e0fe303dfb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14632025-01-15T02:31:30.171ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ca6670c-b2e7-4395-95b0-70e0fe303dfb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14642025-01-15T02:31:30.173ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2357 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ca6670c-b2e7-4395-95b0-70e0fe303dfb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14652025-01-15T02:31:30.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
14662025-01-15T02:31:30.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
14672025-01-15T02:31:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14682025-01-15T02:31:30.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14692025-01-15T02:31:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14702025-01-15T02:31:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14712025-01-15T02:31:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14722025-01-15T02:31:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14732025-01-15T02:31:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14742025-01-15T02:31:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14752025-01-15T02:31:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14762025-01-15T02:31:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14772025-01-15T02:31:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14782025-01-15T02:31:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
14792025-01-15T02:31:30.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14802025-01-15T02:31:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14812025-01-15T02:31:30.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
14822025-01-15T02:31:30.524ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.115379ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
14832025-01-15T02:31:30.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14842025-01-15T02:31:30.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14852025-01-15T02:31:30.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.806775ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14862025-01-15T02:31:31.194ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3d39b486-0bc4-4ac0-852f-8f53974f405e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14872025-01-15T02:31:31.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3d39b486-0bc4-4ac0-852f-8f53974f405e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14882025-01-15T02:31:31.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2313 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3d39b486-0bc4-4ac0-852f-8f53974f405e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14892025-01-15T02:31:32.216ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7bdf21d4-5171-42d9-829b-cd24daac3082 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14902025-01-15T02:31:32.217ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7bdf21d4-5171-42d9-829b-cd24daac3082 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14912025-01-15T02:31:32.218ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2246 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7bdf21d4-5171-42d9-829b-cd24daac3082 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14922025-01-15T02:31:33.239ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ab99c39-07fc-4a3b-9f13-a8a4e461cc8c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14932025-01-15T02:31:33.239ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ab99c39-07fc-4a3b-9f13-a8a4e461cc8c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14942025-01-15T02:31:33.241ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1819 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ab99c39-07fc-4a3b-9f13-a8a4e461cc8c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14952025-01-15T02:31:34.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1c0ad582-5c10-45c6-88d5-27cfd9e463a4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14962025-01-15T02:31:34.261ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1c0ad582-5c10-45c6-88d5-27cfd9e463a4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14972025-01-15T02:31:34.263ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1928 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1c0ad582-5c10-45c6-88d5-27cfd9e463a4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14982025-01-15T02:31:35.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b0c74f3a-8f4d-46c6-ba68-017f270161e4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
14992025-01-15T02:31:35.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b0c74f3a-8f4d-46c6-ba68-017f270161e4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15002025-01-15T02:31:35.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2099 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b0c74f3a-8f4d-46c6-ba68-017f270161e4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15012025-01-15T02:31:35.749ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 609 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 0faacfef-5566-4ff1-a6fe-19708244d9c6 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
15022025-01-15T02:31:36.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f37c013-fe6f-4aa8-acb0-863ee61ba0c1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15032025-01-15T02:31:36.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f37c013-fe6f-4aa8-acb0-863ee61ba0c1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15042025-01-15T02:31:36.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f37c013-fe6f-4aa8-acb0-863ee61ba0c1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15052025-01-15T02:31:37.325ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3caa74df-1d09-46c0-8d04-1493de9a06b7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15062025-01-15T02:31:37.326ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3caa74df-1d09-46c0-8d04-1493de9a06b7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15072025-01-15T02:31:37.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2475 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3caa74df-1d09-46c0-8d04-1493de9a06b7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15082025-01-15T02:31:38.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = befa14ee-b3a2-418f-9b6e-f8cf09e8d903 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15092025-01-15T02:31:38.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = befa14ee-b3a2-418f-9b6e-f8cf09e8d903 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15102025-01-15T02:31:38.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1628 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = befa14ee-b3a2-418f-9b6e-f8cf09e8d903 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15112025-01-15T02:31:39.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3e639c3-7a3c-4d8e-b1f9-c3f89c2a158e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15122025-01-15T02:31:39.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3e639c3-7a3c-4d8e-b1f9-c3f89c2a158e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15132025-01-15T02:31:39.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2339 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3e639c3-7a3c-4d8e-b1f9-c3f89c2a158e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15142025-01-15T02:31:40.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2b0ad585-f0e4-4d6a-b8ec-9e3fdf2cc6ee roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15152025-01-15T02:31:40.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2b0ad585-f0e4-4d6a-b8ec-9e3fdf2cc6ee resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15162025-01-15T02:31:40.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2050 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2b0ad585-f0e4-4d6a-b8ec-9e3fdf2cc6ee response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
15172025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
15182025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
15192025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15202025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
15212025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15222025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
15232025-01-15T02:31:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15242025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
15252025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
15262025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
15272025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15282025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
15292025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15302025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15312025-01-15T02:31:40.507ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15322025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15332025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
15342025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15352025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
15362025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15372025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
15382025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
15392025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15402025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15412025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
15422025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15432025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15442025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
15452025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15462025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
15472025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15482025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15492025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15502025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15512025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15522025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15532025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15542025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15552025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15562025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15572025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15582025-01-15T02:31:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15592025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
15602025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
15612025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15622025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15632025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
15642025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15652025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
15662025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15672025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15682025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15692025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15702025-01-15T02:31:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15712025-01-15T02:31:40.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3587, tv_nsec: 344208283 } })
15722025-01-15T02:31:40.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
15732025-01-15T02:31:40.509ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.387752ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
15742025-01-15T02:31:40.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3587, tv_nsec: 344768078 } })
15752025-01-15T02:31:40.510ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
15762025-01-15T02:31:40.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15772025-01-15T02:31:40.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15782025-01-15T02:31:40.511ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15792025-01-15T02:31:40.512ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15802025-01-15T02:31:40.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15812025-01-15T02:31:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15822025-01-15T02:31:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15832025-01-15T02:31:40.513ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15842025-01-15T02:31:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15852025-01-15T02:31:40.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15862025-01-15T02:31:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15872025-01-15T02:31:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15882025-01-15T02:31:40.516ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15892025-01-15T02:31:40.517ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15902025-01-15T02:31:40.517ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15912025-01-15T02:31:40.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15922025-01-15T02:31:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15932025-01-15T02:31:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15942025-01-15T02:31:40.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15952025-01-15T02:31:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15962025-01-15T02:31:40.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15972025-01-15T02:31:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15982025-01-15T02:31:40.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15992025-01-15T02:31:40.553ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16002025-01-15T02:31:40.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16012025-01-15T02:31:40.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16022025-01-15T02:31:40.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16032025-01-15T02:31:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16042025-01-15T02:31:40.557ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16052025-01-15T02:31:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16062025-01-15T02:31:40.559ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16072025-01-15T02:31:40.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16082025-01-15T02:31:40.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16092025-01-15T02:31:40.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16102025-01-15T02:31:40.560ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16112025-01-15T02:31:40.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16122025-01-15T02:31:40.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16132025-01-15T02:31:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16142025-01-15T02:31:40.562ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16152025-01-15T02:31:40.564ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16162025-01-15T02:31:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16172025-01-15T02:31:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16182025-01-15T02:31:40.564ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16192025-01-15T02:31:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16202025-01-15T02:31:40.566ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16212025-01-15T02:31:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16222025-01-15T02:31:40.568ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16232025-01-15T02:31:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16242025-01-15T02:31:40.570ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3587, tv_nsec: 405202978 } })
16252025-01-15T02:31:40.570ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
16262025-01-15T02:31:40.572ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16272025-01-15T02:31:40.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16282025-01-15T02:31:40.574ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16292025-01-15T02:31:40.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16302025-01-15T02:31:40.576ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16312025-01-15T02:31:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16322025-01-15T02:31:40.577ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 69.329341ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
16332025-01-15T02:31:40.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16342025-01-15T02:31:40.581ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.943388ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
16352025-01-15T02:31:40.581ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16362025-01-15T02:31:40.581ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 74.731647ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16372025-01-15T02:31:40.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16382025-01-15T02:31:40.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16392025-01-15T02:31:40.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16402025-01-15T02:31:40.583ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16412025-01-15T02:31:40.583ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16422025-01-15T02:31:40.583ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16432025-01-15T02:31:40.583ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16442025-01-15T02:31:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16452025-01-15T02:31:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16462025-01-15T02:31:40.591ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 84.170122ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16472025-01-15T02:31:40.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16482025-01-15T02:31:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
16492025-01-15T02:31:40.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16502025-01-15T02:31:40.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16512025-01-15T02:31:40.596ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.346144ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16522025-01-15T02:31:40.604ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 97.702297ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16532025-01-15T02:31:40.619ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
16542025-01-15T02:31:40.620ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 112.966518ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
16552025-01-15T02:31:40.621ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16562025-01-15T02:31:40.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16572025-01-15T02:31:40.622ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16582025-01-15T02:31:40.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
16592025-01-15T02:31:40.624ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16602025-01-15T02:31:40.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16612025-01-15T02:31:40.627ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 120.205273ms iteration = 10 status = 0
16622025-01-15T02:31:40.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16632025-01-15T02:31:40.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16642025-01-15T02:31:40.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16652025-01-15T02:31:40.630ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16662025-01-15T02:31:40.630ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16672025-01-15T02:31:40.630ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 123.335018ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16682025-01-15T02:31:40.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16692025-01-15T02:31:40.648ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16702025-01-15T02:31:40.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16712025-01-15T02:31:40.652ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 143.78232ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16722025-01-15T02:31:40.659ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
16732025-01-15T02:31:40.659ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 152.538036ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16742025-01-15T02:31:40.668ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
16752025-01-15T02:31:40.668ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16762025-01-15T02:31:40.668ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "19f29062-bd01-438e-9018-5947dd9df253", "content-length": "26", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
16772025-01-15T02:31:40.668ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16782025-01-15T02:31:40.669ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
16792025-01-15T02:31:40.676ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 169.235647ms iteration = 12 status = {"physical_disks_added":0}
16802025-01-15T02:31:40.684ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
16812025-01-15T02:31:40.684ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16822025-01-15T02:31:40.684ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "173ee773-400e-41ae-967f-e3a857479a9f", "content-length": "26", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
16832025-01-15T02:31:40.684ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16842025-01-15T02:31:40.687ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16852025-01-15T02:31:40.700ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
16862025-01-15T02:31:40.700ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16872025-01-15T02:31:40.700ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab13ada4-fa50-45f0-9572-35085bbba56e", "content-length": "26", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
16882025-01-15T02:31:40.700ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16892025-01-15T02:31:40.701ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16902025-01-15T02:31:40.749ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16912025-01-15T02:31:40.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16922025-01-15T02:31:40.751ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16932025-01-15T02:31:40.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16942025-01-15T02:31:40.753ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16952025-01-15T02:31:40.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16962025-01-15T02:31:40.754ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16972025-01-15T02:31:40.756ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16982025-01-15T02:31:40.756ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16992025-01-15T02:31:40.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17002025-01-15T02:31:40.776ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
17012025-01-15T02:31:40.776ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17022025-01-15T02:31:40.822ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
17032025-01-15T02:31:40.822ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4cae49b-a25e-43fe-a82f-f282b1a40789", "content-length": "1", "date": "Wed, 15 Jan 2025 02:31:39 GMT"} })
17042025-01-15T02:31:40.823ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17052025-01-15T02:31:40.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17062025-01-15T02:31:40.826ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 319.622005ms iteration = 10 status = null
17072025-01-15T02:31:40.868ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e9b35a4-e383-46d0-8521-b733d6891620", "content-length": "372", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17082025-01-15T02:31:40.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17092025-01-15T02:31:40.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17102025-01-15T02:31:40.883ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
17112025-01-15T02:31:40.883ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17122025-01-15T02:31:40.883ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "620db641-e195-42a9-978e-de36404f117d", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17132025-01-15T02:31:40.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17142025-01-15T02:31:40.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17152025-01-15T02:31:40.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17162025-01-15T02:31:40.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17172025-01-15T02:31:40.906ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17182025-01-15T02:31:40.906ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "09ab70ea-1daf-410c-b66e-d772eaa9edd4", "content-length": "691", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17192025-01-15T02:31:40.906ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17202025-01-15T02:31:40.906ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17212025-01-15T02:31:40.906ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17222025-01-15T02:31:40.908ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 401.186896ms iteration = 10 status = {}
17232025-01-15T02:31:40.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17242025-01-15T02:31:40.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17252025-01-15T02:31:40.910ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 404.033473ms iteration = 10 status = {}
17262025-01-15T02:31:40.912ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17272025-01-15T02:31:40.912ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17282025-01-15T02:31:40.912ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "af0d09cb-4c28-430f-8a48-6c8fe58e3b61", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17292025-01-15T02:31:40.914ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 407.378923ms iteration = 10 status = {}
17302025-01-15T02:31:40.966ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17312025-01-15T02:31:40.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17322025-01-15T02:31:40.968ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
17332025-01-15T02:31:40.968ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17342025-01-15T02:31:40.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17352025-01-15T02:31:40.977ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
17362025-01-15T02:31:40.977ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6da02df7-7fe2-4665-9ecb-7bbd88f0f371", "content-length": "134", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17372025-01-15T02:31:40.977ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
17382025-01-15T02:31:40.977ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
17392025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17402025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17412025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17422025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17432025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17442025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17452025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17462025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17472025-01-15T02:31:40.978ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
17482025-01-15T02:31:40.978ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17492025-01-15T02:31:40.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17502025-01-15T02:31:40.980ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
17512025-01-15T02:31:40.980ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e0cd93f-61a4-459a-965b-be5c9ebff132", "content-length": "2", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17522025-01-15T02:31:40.980ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
17532025-01-15T02:31:40.980ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d4681ff-4a4f-4ef7-a9d6-79db7df3f6e6", "content-length": "52", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17542025-01-15T02:31:40.980ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
17552025-01-15T02:31:40.980ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17562025-01-15T02:31:41.076ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "55c9e636-717f-40a7-9c15-6687850b97c0", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17572025-01-15T02:31:41.076ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
17582025-01-15T02:31:41.076ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17592025-01-15T02:31:41.076ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "58422ff7-9488-4379-ba77-78dedbc896cd", "content-length": "682", "date": "Wed, 15 Jan 2025 02:31:40 GMT"} })
17602025-01-15T02:31:41.076ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17612025-01-15T02:31:41.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17622025-01-15T02:31:41.080ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17632025-01-15T02:31:41.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17642025-01-15T02:31:41.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17652025-01-15T02:31:41.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17662025-01-15T02:31:41.086ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17672025-01-15T02:31:41.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17682025-01-15T02:31:41.094ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 587.363449ms iteration = 10 status = {}
17692025-01-15T02:31:41.413ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df97b3fe-5b28-4d66-976b-2ecf22937aa5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17702025-01-15T02:31:41.414ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df97b3fe-5b28-4d66-976b-2ecf22937aa5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17712025-01-15T02:31:41.416ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2430 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df97b3fe-5b28-4d66-976b-2ecf22937aa5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17722025-01-15T02:31:42.435ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b44044b6-4950-44b4-b14f-2ded0a3708cf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17732025-01-15T02:31:42.436ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b44044b6-4950-44b4-b14f-2ded0a3708cf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17742025-01-15T02:31:42.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2048 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b44044b6-4950-44b4-b14f-2ded0a3708cf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17752025-01-15T02:31:43.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a88a745b-7c1a-4f82-84aa-46008a245be0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17762025-01-15T02:31:43.458ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a88a745b-7c1a-4f82-84aa-46008a245be0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17772025-01-15T02:31:43.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2312 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a88a745b-7c1a-4f82-84aa-46008a245be0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17782025-01-15T02:31:44.479ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63bebaa9-a5c4-4021-935b-5952000a6ef8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17792025-01-15T02:31:44.480ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63bebaa9-a5c4-4021-935b-5952000a6ef8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17802025-01-15T02:31:44.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63bebaa9-a5c4-4021-935b-5952000a6ef8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17812025-01-15T02:31:45.500ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cc69cc36-e078-4702-89c2-9d974e59f4f4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17822025-01-15T02:31:45.500ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cc69cc36-e078-4702-89c2-9d974e59f4f4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17832025-01-15T02:31:45.502ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2347 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cc69cc36-e078-4702-89c2-9d974e59f4f4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17842025-01-15T02:31:45.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 444 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = e50e1aa3-8426-43a2-aa26-4f366319da8e response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
17852025-01-15T02:31:46.522ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bbbacf99-88f6-49d3-8c82-91ae1241b503 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17862025-01-15T02:31:46.522ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bbbacf99-88f6-49d3-8c82-91ae1241b503 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17872025-01-15T02:31:46.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2203 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bbbacf99-88f6-49d3-8c82-91ae1241b503 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17882025-01-15T02:31:47.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5896eee0-33ab-425d-bf20-2f14bca9ab7b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17892025-01-15T02:31:47.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5896eee0-33ab-425d-bf20-2f14bca9ab7b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17902025-01-15T02:31:47.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2309 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5896eee0-33ab-425d-bf20-2f14bca9ab7b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17912025-01-15T02:31:48.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 01b8dc7c-3e66-4ac0-89eb-d4d1ba3ce52f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17922025-01-15T02:31:48.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 01b8dc7c-3e66-4ac0-89eb-d4d1ba3ce52f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17932025-01-15T02:31:48.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2007 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 01b8dc7c-3e66-4ac0-89eb-d4d1ba3ce52f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17942025-01-15T02:31:49.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c407a87-83cc-4c33-bb4b-05ece9213909 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17952025-01-15T02:31:49.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c407a87-83cc-4c33-bb4b-05ece9213909 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17962025-01-15T02:31:49.590ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2012 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c407a87-83cc-4c33-bb4b-05ece9213909 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
17972025-01-15T02:31:50.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
17982025-01-15T02:31:50.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
17992025-01-15T02:31:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18002025-01-15T02:31:50.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18012025-01-15T02:31:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18022025-01-15T02:31:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18032025-01-15T02:31:50.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18042025-01-15T02:31:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18052025-01-15T02:31:50.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18062025-01-15T02:31:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18072025-01-15T02:31:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18082025-01-15T02:31:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18092025-01-15T02:31:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18102025-01-15T02:31:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18112025-01-15T02:31:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18122025-01-15T02:31:50.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
18132025-01-15T02:31:50.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18142025-01-15T02:31:50.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18152025-01-15T02:31:50.528ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
18162025-01-15T02:31:50.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.693095ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
18172025-01-15T02:31:50.543ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.977962ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18182025-01-15T02:31:50.612ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 091d7393-65ab-4721-875f-06cd5165d091 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18192025-01-15T02:31:50.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 091d7393-65ab-4721-875f-06cd5165d091 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18202025-01-15T02:31:50.614ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1962 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 091d7393-65ab-4721-875f-06cd5165d091 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18212025-01-15T02:31:51.634ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a45df2d-4cb7-4cfa-b365-a152066c8bf1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18222025-01-15T02:31:51.634ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a45df2d-4cb7-4cfa-b365-a152066c8bf1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18232025-01-15T02:31:51.636ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1704 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a45df2d-4cb7-4cfa-b365-a152066c8bf1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18242025-01-15T02:31:52.656ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 09b63cb3-fdc7-4fed-b4c5-aa46b73c0b6f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18252025-01-15T02:31:52.657ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 09b63cb3-fdc7-4fed-b4c5-aa46b73c0b6f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18262025-01-15T02:31:52.658ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2291 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 09b63cb3-fdc7-4fed-b4c5-aa46b73c0b6f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18272025-01-15T02:31:53.677ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0fb432e5-b76e-4c2c-b43e-3a0241a0ff36 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18282025-01-15T02:31:53.678ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0fb432e5-b76e-4c2c-b43e-3a0241a0ff36 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18292025-01-15T02:31:53.680ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2260 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0fb432e5-b76e-4c2c-b43e-3a0241a0ff36 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18302025-01-15T02:31:54.699ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 56f449d2-31b2-4132-8967-7847e9197f3c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18312025-01-15T02:31:54.699ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 56f449d2-31b2-4132-8967-7847e9197f3c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18322025-01-15T02:31:54.701ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2029 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 56f449d2-31b2-4132-8967-7847e9197f3c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18332025-01-15T02:31:55.720ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf5a1097-98f9-44d7-8a72-3d9281589610 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18342025-01-15T02:31:55.720ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf5a1097-98f9-44d7-8a72-3d9281589610 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18352025-01-15T02:31:55.722ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2110 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf5a1097-98f9-44d7-8a72-3d9281589610 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18362025-01-15T02:31:55.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 406 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = d99ea00e-74c3-4080-b219-17f040968460 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
18372025-01-15T02:31:56.742ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1309660d-db11-45fc-b812-d7cfa49d8300 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18382025-01-15T02:31:56.742ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1309660d-db11-45fc-b812-d7cfa49d8300 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18392025-01-15T02:31:56.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1605 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1309660d-db11-45fc-b812-d7cfa49d8300 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18402025-01-15T02:31:57.764ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 15acb94f-1045-427c-8477-5afad226bf29 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18412025-01-15T02:31:57.764ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 15acb94f-1045-427c-8477-5afad226bf29 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18422025-01-15T02:31:57.766ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2035 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 15acb94f-1045-427c-8477-5afad226bf29 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18432025-01-15T02:31:58.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df5aa11d-66b6-4898-85ba-43192151779c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18442025-01-15T02:31:58.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df5aa11d-66b6-4898-85ba-43192151779c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18452025-01-15T02:31:58.789ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2211 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df5aa11d-66b6-4898-85ba-43192151779c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18462025-01-15T02:31:59.808ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d684f703-b2be-4e0d-a4b0-c45f3fb3f1f0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18472025-01-15T02:31:59.808ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d684f703-b2be-4e0d-a4b0-c45f3fb3f1f0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18482025-01-15T02:31:59.810ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d684f703-b2be-4e0d-a4b0-c45f3fb3f1f0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18492025-01-15T02:32:00.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
18502025-01-15T02:32:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18512025-01-15T02:32:00.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
18522025-01-15T02:32:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18532025-01-15T02:32:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18542025-01-15T02:32:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18552025-01-15T02:32:00.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18562025-01-15T02:32:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18572025-01-15T02:32:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18582025-01-15T02:32:00.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.30244ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18592025-01-15T02:32:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18602025-01-15T02:32:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18612025-01-15T02:32:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18622025-01-15T02:32:00.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18632025-01-15T02:32:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18642025-01-15T02:32:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18652025-01-15T02:32:00.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
18662025-01-15T02:32:00.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18672025-01-15T02:32:00.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18682025-01-15T02:32:00.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
18692025-01-15T02:32:00.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.415711ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
18702025-01-15T02:32:00.830ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ff5a8f47-ae49-4ccf-a873-f6cc7838397f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18712025-01-15T02:32:00.831ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ff5a8f47-ae49-4ccf-a873-f6cc7838397f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18722025-01-15T02:32:00.832ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2030 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ff5a8f47-ae49-4ccf-a873-f6cc7838397f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18732025-01-15T02:32:01.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e3d988b-7e89-4e0b-babf-479893e2eae9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18742025-01-15T02:32:01.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e3d988b-7e89-4e0b-babf-479893e2eae9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18752025-01-15T02:32:01.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2166 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e3d988b-7e89-4e0b-babf-479893e2eae9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18762025-01-15T02:32:02.876ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4e55a076-f129-4a16-b251-1291b34e0a0e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18772025-01-15T02:32:02.876ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4e55a076-f129-4a16-b251-1291b34e0a0e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18782025-01-15T02:32:02.878ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2223 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4e55a076-f129-4a16-b251-1291b34e0a0e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18792025-01-15T02:32:03.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae7b176f-6ad4-48a6-981e-bf4956216885 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18802025-01-15T02:32:03.898ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae7b176f-6ad4-48a6-981e-bf4956216885 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18812025-01-15T02:32:03.900ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2416 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae7b176f-6ad4-48a6-981e-bf4956216885 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18822025-01-15T02:32:04.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4f598c3-345a-45c5-b34d-b479f2fe5bf7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18832025-01-15T02:32:04.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4f598c3-345a-45c5-b34d-b479f2fe5bf7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18842025-01-15T02:32:04.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4f598c3-345a-45c5-b34d-b479f2fe5bf7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18852025-01-15T02:32:05.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 530 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 803a57b1-971e-4327-807d-061f308841b2 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
18862025-01-15T02:32:05.942ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e3aaba5b-a2b3-43af-9463-4fc901258f4e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18872025-01-15T02:32:05.942ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e3aaba5b-a2b3-43af-9463-4fc901258f4e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18882025-01-15T02:32:05.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e3aaba5b-a2b3-43af-9463-4fc901258f4e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18892025-01-15T02:32:06.964ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 78539e42-b99b-4963-beae-e38c8541f2fa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18902025-01-15T02:32:06.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 78539e42-b99b-4963-beae-e38c8541f2fa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18912025-01-15T02:32:06.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2635 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 78539e42-b99b-4963-beae-e38c8541f2fa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18922025-01-15T02:32:07.987ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ccf08ce-2df7-4047-9097-7da7a5b23a6c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18932025-01-15T02:32:07.988ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ccf08ce-2df7-4047-9097-7da7a5b23a6c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18942025-01-15T02:32:07.990ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2321 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ccf08ce-2df7-4047-9097-7da7a5b23a6c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18952025-01-15T02:32:09.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = caaf28a8-a70f-4e90-a388-e294daa03c6d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18962025-01-15T02:32:09.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = caaf28a8-a70f-4e90-a388-e294daa03c6d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18972025-01-15T02:32:09.013ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2094 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = caaf28a8-a70f-4e90-a388-e294daa03c6d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18982025-01-15T02:32:10.032ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 55eaa2da-170a-4f9c-a22c-e99f40fc1760 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
18992025-01-15T02:32:10.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 55eaa2da-170a-4f9c-a22c-e99f40fc1760 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
19002025-01-15T02:32:10.034ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2323 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 55eaa2da-170a-4f9c-a22c-e99f40fc1760 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
19012025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
19022025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
19032025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
19042025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
19052025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
19062025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
19072025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19082025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19092025-01-15T02:32:10.505ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
19102025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
19112025-01-15T02:32:10.506ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:22:10.505976941 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
19122025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19132025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
19142025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
19152025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
19162025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
19172025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19182025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19192025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19202025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19212025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19222025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19232025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19242025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19252025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
19262025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
19272025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
19282025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
19292025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19302025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
19312025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19322025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19332025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19342025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Timeout
19352025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19362025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
19372025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19382025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19392025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
19402025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
19412025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
19422025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19432025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19442025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
19452025-01-15T02:32:10.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19462025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19472025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
19482025-01-15T02:32:10.506ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
19492025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.561\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
19502025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
19512025-01-15T02:32:10.506ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
19522025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
19532025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19542025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19552025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
19562025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
19572025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
19582025-01-15T02:32:10.506ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
19592025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19602025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
19612025-01-15T02:32:10.506ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3617, tv_nsec: 341800708 } })
19622025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19632025-01-15T02:32:10.506ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3617, tv_nsec: 341824419 } })
19642025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19652025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19662025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19672025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19682025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19692025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19702025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19712025-01-15T02:32:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19722025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19732025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19742025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
19752025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19762025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19772025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
19782025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
19792025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.308075ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
19802025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
19812025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.357087ms iteration = 6 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
19822025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19832025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
19842025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
19852025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
19862025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19872025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
19882025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19892025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
19902025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
19912025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
19922025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19932025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
19942025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19952025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19962025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
19972025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19982025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
19992025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20002025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20012025-01-15T02:32:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20022025-01-15T02:32:10.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3617, tv_nsec: 344541752 } })
20032025-01-15T02:32:10.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
20042025-01-15T02:32:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.283086ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20052025-01-15T02:32:10.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20062025-01-15T02:32:10.510ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3617, tv_nsec: 345654002 } })
20072025-01-15T02:32:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20082025-01-15T02:32:10.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20092025-01-15T02:32:10.510ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
20102025-01-15T02:32:10.510ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:22:10.505976941 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
20112025-01-15T02:32:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.83717ms iteration = 6 status = {"expiration":"2025-01-15T02:22:10.505976941Z","pruned":[]}
20122025-01-15T02:32:10.512ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20132025-01-15T02:32:10.512ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20142025-01-15T02:32:10.512ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20152025-01-15T02:32:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20162025-01-15T02:32:10.514ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20172025-01-15T02:32:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20182025-01-15T02:32:10.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20192025-01-15T02:32:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20202025-01-15T02:32:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20212025-01-15T02:32:10.514ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20222025-01-15T02:32:10.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20232025-01-15T02:32:10.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20242025-01-15T02:32:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20252025-01-15T02:32:10.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20262025-01-15T02:32:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20272025-01-15T02:32:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20282025-01-15T02:32:10.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20292025-01-15T02:32:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20302025-01-15T02:32:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20312025-01-15T02:32:10.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20322025-01-15T02:32:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20332025-01-15T02:32:10.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20342025-01-15T02:32:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20352025-01-15T02:32:10.523ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
20362025-01-15T02:32:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
20372025-01-15T02:32:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.747709ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
20382025-01-15T02:32:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 19.694792ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
20392025-01-15T02:32:10.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20402025-01-15T02:32:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20412025-01-15T02:32:10.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20422025-01-15T02:32:10.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20432025-01-15T02:32:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20442025-01-15T02:32:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20452025-01-15T02:32:10.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20462025-01-15T02:32:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20472025-01-15T02:32:10.558ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20482025-01-15T02:32:10.558ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20492025-01-15T02:32:10.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20502025-01-15T02:32:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
20512025-01-15T02:32:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20522025-01-15T02:32:10.561ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20532025-01-15T02:32:10.561ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20542025-01-15T02:32:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20552025-01-15T02:32:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20562025-01-15T02:32:10.563ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20572025-01-15T02:32:10.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20582025-01-15T02:32:10.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20592025-01-15T02:32:10.564ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20602025-01-15T02:32:10.564ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
20612025-01-15T02:32:10.564ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20622025-01-15T02:32:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
20632025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20642025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20652025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20662025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20672025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20682025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20692025-01-15T02:32:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20702025-01-15T02:32:10.568ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20712025-01-15T02:32:10.568ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20722025-01-15T02:32:10.568ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20732025-01-15T02:32:10.568ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20742025-01-15T02:32:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20752025-01-15T02:32:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20762025-01-15T02:32:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20772025-01-15T02:32:10.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20782025-01-15T02:32:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20792025-01-15T02:32:10.570ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.531054ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
20802025-01-15T02:32:10.571ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
20812025-01-15T02:32:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
20822025-01-15T02:32:10.572ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20832025-01-15T02:32:10.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20842025-01-15T02:32:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20852025-01-15T02:32:10.572ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20862025-01-15T02:32:10.573ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.807873ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20872025-01-15T02:32:10.574ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
20882025-01-15T02:32:10.574ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
20892025-01-15T02:32:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 66.828533ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
20902025-01-15T02:32:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
20912025-01-15T02:32:10.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20922025-01-15T02:32:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20932025-01-15T02:32:10.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20942025-01-15T02:32:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20952025-01-15T02:32:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20962025-01-15T02:32:10.576ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20972025-01-15T02:32:10.577ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.543933ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20982025-01-15T02:32:10.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20992025-01-15T02:32:10.578ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21002025-01-15T02:32:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21012025-01-15T02:32:10.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21022025-01-15T02:32:10.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21032025-01-15T02:32:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21042025-01-15T02:32:10.591ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.188638ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21052025-01-15T02:32:10.595ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3617, tv_nsec: 430524422 } })
21062025-01-15T02:32:10.595ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
21072025-01-15T02:32:10.608ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
21082025-01-15T02:32:10.608ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 101.90482ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
21092025-01-15T02:32:10.615ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
21102025-01-15T02:32:10.615ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21112025-01-15T02:32:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
21122025-01-15T02:32:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21132025-01-15T02:32:10.618ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21142025-01-15T02:32:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21152025-01-15T02:32:10.619ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21162025-01-15T02:32:10.619ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21172025-01-15T02:32:10.619ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21182025-01-15T02:32:10.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21192025-01-15T02:32:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21202025-01-15T02:32:10.620ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21212025-01-15T02:32:10.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21222025-01-15T02:32:10.621ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21232025-01-15T02:32:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21242025-01-15T02:32:10.622ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21252025-01-15T02:32:10.622ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
21262025-01-15T02:32:10.622ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 116.162044ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21272025-01-15T02:32:10.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
21282025-01-15T02:32:10.623ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21292025-01-15T02:32:10.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21302025-01-15T02:32:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21312025-01-15T02:32:10.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21322025-01-15T02:32:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21332025-01-15T02:32:10.626ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21342025-01-15T02:32:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21352025-01-15T02:32:10.627ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 120.031869ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21362025-01-15T02:32:10.628ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 122.007992ms iteration = 11 status = 0
21372025-01-15T02:32:10.628ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21382025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21392025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21402025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21412025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21422025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21432025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21442025-01-15T02:32:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21452025-01-15T02:32:10.630ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
21462025-01-15T02:32:10.631ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21472025-01-15T02:32:10.631ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21482025-01-15T02:32:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21492025-01-15T02:32:10.632ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
21502025-01-15T02:32:10.632ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21512025-01-15T02:32:10.632ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "90ec186f-0195-40b2-b836-d52369ca00ec", "content-length": "26", "date": "Wed, 15 Jan 2025 02:32:09 GMT"} })
21522025-01-15T02:32:10.632ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21532025-01-15T02:32:10.635ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21542025-01-15T02:32:10.637ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21552025-01-15T02:32:10.637ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
21562025-01-15T02:32:10.637ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21572025-01-15T02:32:10.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21582025-01-15T02:32:10.639ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21592025-01-15T02:32:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21602025-01-15T02:32:10.666ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21612025-01-15T02:32:10.667ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21622025-01-15T02:32:10.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21632025-01-15T02:32:10.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
21642025-01-15T02:32:10.669ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21652025-01-15T02:32:10.669ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21662025-01-15T02:32:10.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21672025-01-15T02:32:10.671ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
21682025-01-15T02:32:10.671ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 165.044934ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
21692025-01-15T02:32:10.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21702025-01-15T02:32:10.672ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21712025-01-15T02:32:10.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21722025-01-15T02:32:10.675ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 167.867769ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
21732025-01-15T02:32:10.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21742025-01-15T02:32:10.678ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
21752025-01-15T02:32:10.678ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
21762025-01-15T02:32:10.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21772025-01-15T02:32:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21782025-01-15T02:32:10.680ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21792025-01-15T02:32:10.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21802025-01-15T02:32:10.680ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21812025-01-15T02:32:10.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21822025-01-15T02:32:10.681ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21832025-01-15T02:32:10.681ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21842025-01-15T02:32:10.681ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 175.108085ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21852025-01-15T02:32:10.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21862025-01-15T02:32:10.682ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21872025-01-15T02:32:10.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21882025-01-15T02:32:10.685ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21892025-01-15T02:32:10.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21902025-01-15T02:32:10.685ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
21912025-01-15T02:32:10.685ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21922025-01-15T02:32:10.685ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
21932025-01-15T02:32:10.685ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 179.670488ms iteration = 6 status = {"generation":2}
21942025-01-15T02:32:10.686ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21952025-01-15T02:32:10.687ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
21962025-01-15T02:32:10.687ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 180.427858ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
21972025-01-15T02:32:10.687ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
21982025-01-15T02:32:10.687ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 181.836707ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
21992025-01-15T02:32:10.688ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22002025-01-15T02:32:10.688ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22012025-01-15T02:32:10.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22022025-01-15T02:32:10.690ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22032025-01-15T02:32:10.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22042025-01-15T02:32:10.693ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22052025-01-15T02:32:10.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22062025-01-15T02:32:10.709ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22072025-01-15T02:32:10.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22082025-01-15T02:32:10.711ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22092025-01-15T02:32:10.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22102025-01-15T02:32:10.719ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 213.002758ms iteration = 13 status = {"physical_disks_added":0}
22112025-01-15T02:32:10.720ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22122025-01-15T02:32:10.720ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22132025-01-15T02:32:10.720ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22142025-01-15T02:32:10.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22152025-01-15T02:32:10.743ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22162025-01-15T02:32:10.748ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
22172025-01-15T02:32:10.748ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22182025-01-15T02:32:10.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22192025-01-15T02:32:10.759ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
22202025-01-15T02:32:10.759ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22212025-01-15T02:32:10.759ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "17a995fa-b9bb-4879-a5c1-e636ea84eb28", "content-length": "26", "date": "Wed, 15 Jan 2025 02:32:09 GMT"} })
22222025-01-15T02:32:10.759ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22232025-01-15T02:32:10.761ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
22242025-01-15T02:32:10.769ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22252025-01-15T02:32:10.814ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22262025-01-15T02:32:10.815ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7a74ea3-5702-46e6-9950-d6382b88806e", "content-length": "1", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} })
22272025-01-15T02:32:10.817ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22282025-01-15T02:32:10.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22292025-01-15T02:32:10.834ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
22302025-01-15T02:32:10.834ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22312025-01-15T02:32:10.834ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e9fecb5-f939-4f3a-8cfe-6b8de65b6f08", "content-length": "26", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} })
22322025-01-15T02:32:10.834ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22332025-01-15T02:32:10.837ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22342025-01-15T02:32:10.929ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22352025-01-15T02:32:10.979ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22362025-01-15T02:32:10.979ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "9bd99f09-b18c-48ad-bbb8-34dba18e60e3", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22372025-01-15T02:32:10.979ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22382025-01-15T02:32:10.980ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "baece8a9-18ee-4277-bf5d-a68e90dd8936", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22392025-01-15T02:32:10.980ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22402025-01-15T02:32:10.982ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22412025-01-15T02:32:10.983ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 477.094784ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22422025-01-15T02:32:11.006ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
22432025-01-15T02:32:11.006ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22442025-01-15T02:32:11.007ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 501.271046ms iteration = 11 status = {}
22452025-01-15T02:32:11.007ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22462025-01-15T02:32:11.010ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22472025-01-15T02:32:11.010ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22482025-01-15T02:32:11.010ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22492025-01-15T02:32:11.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22502025-01-15T02:32:11.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22512025-01-15T02:32:11.046ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
22522025-01-15T02:32:11.048ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbde6deb-0d6f-4e67-bf2c-20ecc07c1315", "content-length": "691", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} })
22532025-01-15T02:32:11.048ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "349271b6-3a48-40db-8fef-2b9f59d30169", "content-length": "372", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
22542025-01-15T02:32:11.048ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
22552025-01-15T02:32:11.048ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22562025-01-15T02:32:11.048ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
22572025-01-15T02:32:11.048ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
22582025-01-15T02:32:11.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22592025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 543.437154ms iteration = 11 status = null
22602025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22612025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22622025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
22632025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22642025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
22652025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 543.958139ms iteration = 6 status = {"generation":1}
22662025-01-15T02:32:11.049ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 543.488595ms iteration = 11 status = {}
22672025-01-15T02:32:11.051ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22682025-01-15T02:32:11.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22692025-01-15T02:32:11.054ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22702025-01-15T02:32:11.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22712025-01-15T02:32:11.055ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22722025-01-15T02:32:11.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22732025-01-15T02:32:11.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 85ae4eab-d43e-481d-be2a-d6c00b76a592 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
22742025-01-15T02:32:11.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 85ae4eab-d43e-481d-be2a-d6c00b76a592 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
22752025-01-15T02:32:11.061ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22762025-01-15T02:32:11.062ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2bf33bd8-426f-4758-94cd-3d6612386450", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22772025-01-15T02:32:11.062ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22782025-01-15T02:32:11.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2363 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 85ae4eab-d43e-481d-be2a-d6c00b76a592 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
22792025-01-15T02:32:11.063ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "40b36fe4-ccb9-4bea-952d-93a12cf02320", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22802025-01-15T02:32:11.063ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22812025-01-15T02:32:11.065ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0e684f88-a8fa-4a49-ac02-bade6ea59d09", "date": "Wed, 15 Jan 2025 02:32:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22822025-01-15T02:32:11.065ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22832025-01-15T02:32:11.067ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22842025-01-15T02:32:11.067ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 561.672946ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
22852025-01-15T02:32:11.084ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22862025-01-15T02:32:11.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22872025-01-15T02:32:11.091ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22882025-01-15T02:32:11.091ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22892025-01-15T02:32:11.091ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4468fb9f-7df3-43a9-a3e4-a2f77d506cef", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
22902025-01-15T02:32:11.091ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22912025-01-15T02:32:11.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22922025-01-15T02:32:11.093ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22932025-01-15T02:32:11.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22942025-01-15T02:32:11.097ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22952025-01-15T02:32:11.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22962025-01-15T02:32:11.100ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22972025-01-15T02:32:11.100ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22982025-01-15T02:32:11.100ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8ea30964-8c65-4e34-80ab-6783e1e82745", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
22992025-01-15T02:32:11.103ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 596.610349ms iteration = 11 status = {}
23002025-01-15T02:32:11.114ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23012025-01-15T02:32:11.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23022025-01-15T02:32:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23032025-01-15T02:32:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
23042025-01-15T02:32:11.117ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23052025-01-15T02:32:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23062025-01-15T02:32:11.118ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
23072025-01-15T02:32:11.118ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23082025-01-15T02:32:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23092025-01-15T02:32:11.120ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23102025-01-15T02:32:11.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23112025-01-15T02:32:11.121ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23122025-01-15T02:32:11.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23132025-01-15T02:32:11.124ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23142025-01-15T02:32:11.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23152025-01-15T02:32:11.126ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23162025-01-15T02:32:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23172025-01-15T02:32:11.127ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23182025-01-15T02:32:11.127ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "9810c29f-1a36-42f6-b842-a1c37d72d38e", "content-length": "134", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
23192025-01-15T02:32:11.127ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
23202025-01-15T02:32:11.127ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
23212025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
23222025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23232025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23242025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23252025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23262025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23272025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23282025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23292025-01-15T02:32:11.128ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
23302025-01-15T02:32:11.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23312025-01-15T02:32:11.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23322025-01-15T02:32:11.130ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23332025-01-15T02:32:11.130ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "796cbbdf-e1ce-4ea8-b793-db299f4cc41a", "content-length": "2", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
23342025-01-15T02:32:11.131ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23352025-01-15T02:32:11.131ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb63659a-a3bc-4031-9bdf-ff1401ba6ee6", "content-length": "52", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
23362025-01-15T02:32:11.131ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
23372025-01-15T02:32:11.131ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23382025-01-15T02:32:11.159ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23392025-01-15T02:32:11.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23402025-01-15T02:32:11.160ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23412025-01-15T02:32:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23422025-01-15T02:32:11.162ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23432025-01-15T02:32:11.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23442025-01-15T02:32:11.166ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23452025-01-15T02:32:11.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23462025-01-15T02:32:11.167ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23472025-01-15T02:32:11.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23482025-01-15T02:32:11.168ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23492025-01-15T02:32:11.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23502025-01-15T02:32:11.197ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23512025-01-15T02:32:11.201ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23522025-01-15T02:32:11.201ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23532025-01-15T02:32:11.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23542025-01-15T02:32:11.205ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23552025-01-15T02:32:11.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23562025-01-15T02:32:11.207ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23572025-01-15T02:32:11.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23582025-01-15T02:32:11.209ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23592025-01-15T02:32:11.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23602025-01-15T02:32:11.211ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23612025-01-15T02:32:11.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23622025-01-15T02:32:11.214ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23632025-01-15T02:32:11.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23642025-01-15T02:32:11.215ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23652025-01-15T02:32:11.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23662025-01-15T02:32:11.217ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23672025-01-15T02:32:11.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23682025-01-15T02:32:11.218ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23692025-01-15T02:32:11.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23702025-01-15T02:32:11.221ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23712025-01-15T02:32:11.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23722025-01-15T02:32:11.223ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23732025-01-15T02:32:11.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23742025-01-15T02:32:11.224ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23752025-01-15T02:32:11.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23762025-01-15T02:32:11.226ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23772025-01-15T02:32:11.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23782025-01-15T02:32:11.232ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23792025-01-15T02:32:11.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23802025-01-15T02:32:11.234ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6ebe13fa-8542-4662-a5a2-4fa9a01824c9", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
23812025-01-15T02:32:11.234ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
23822025-01-15T02:32:11.234ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23832025-01-15T02:32:11.234ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d215ed31-2128-4855-b3c4-0bb4dcc3c0b9", "content-length": "682", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
23842025-01-15T02:32:11.235ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23852025-01-15T02:32:11.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23862025-01-15T02:32:11.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23872025-01-15T02:32:11.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23882025-01-15T02:32:11.238ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: b15378a9-5120-477b-abef-a170753103c1, kind: Service { id: 7468547b-b2f3-4993-afd8-72e9164a4301 }, name: Name("external-dns-7468547b-b2f3-4993-afd8-72e9164a4301"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 124])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: beea2870-fa6b-4d58-b0eb-af5a33149d11, kind: Service { id: 8b302e6b-b09c-4ef2-8768-1e5fae50d31a }, name: Name("external-dns-8b302e6b-b09c-4ef2-8768-1e5fae50d31a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 22195cd0-81ca-4c14-8294-e0bb8767fa51, kind: Service { id: 743f2600-b4a7-4963-8bef-692428b6800a }, name: Name("nexus-743f2600-b4a7-4963-8bef-692428b6800a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 161])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 357305ec-a785-4a6e-96cf-879dc3168076, kind: Service { id: a1e38c68-2155-4ce9-b70a-66d434b3f24f }, name: Name("nexus-a1e38c68-2155-4ce9-b70a-66d434b3f24f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 56f43fd0-1c39-449d-9089-5f10b7acf59b, kind: Service { id: b4bce764-527e-4a0d-9efa-46f5760fbb1d }, name: Name("nexus-b4bce764-527e-4a0d-9efa-46f5760fbb1d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 31])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
23892025-01-15T02:32:11.238ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23902025-01-15T02:32:11.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23912025-01-15T02:32:11.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23922025-01-15T02:32:11.241ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23932025-01-15T02:32:11.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23942025-01-15T02:32:11.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
23952025-01-15T02:32:11.241ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23962025-01-15T02:32:11.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23972025-01-15T02:32:11.243ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
23982025-01-15T02:32:11.243ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23992025-01-15T02:32:11.243ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: b15378a9-5120-477b-abef-a170753103c1, kind: Service { id: 7468547b-b2f3-4993-afd8-72e9164a4301 }, name: Name("external-dns-7468547b-b2f3-4993-afd8-72e9164a4301"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 124])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: beea2870-fa6b-4d58-b0eb-af5a33149d11, kind: Service { id: 8b302e6b-b09c-4ef2-8768-1e5fae50d31a }, name: Name("external-dns-8b302e6b-b09c-4ef2-8768-1e5fae50d31a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 22195cd0-81ca-4c14-8294-e0bb8767fa51, kind: Service { id: 743f2600-b4a7-4963-8bef-692428b6800a }, name: Name("nexus-743f2600-b4a7-4963-8bef-692428b6800a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 161])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 357305ec-a785-4a6e-96cf-879dc3168076, kind: Service { id: a1e38c68-2155-4ce9-b70a-66d434b3f24f }, name: Name("nexus-a1e38c68-2155-4ce9-b70a-66d434b3f24f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 56f43fd0-1c39-449d-9089-5f10b7acf59b, kind: Service { id: b4bce764-527e-4a0d-9efa-46f5760fbb1d }, name: Name("nexus-b4bce764-527e-4a0d-9efa-46f5760fbb1d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 31])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
24002025-01-15T02:32:11.243ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
24012025-01-15T02:32:11.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24022025-01-15T02:32:11.249ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 742.840285ms iteration = 11 status = {}
24032025-01-15T02:32:11.349ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: ce66c06a-7d54-47a6-860d-59da68080fee, time_created: 2025-01-15T02:27:10.189884Z, time_modified: 2025-01-15T02:27:10.189884Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
24042025-01-15T02:32:11.349ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
24052025-01-15T02:32:11.349ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
24062025-01-15T02:32:11.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
24072025-01-15T02:32:11.352ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
24082025-01-15T02:32:11.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
24092025-01-15T02:32:11.354ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24102025-01-15T02:32:11.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
24112025-01-15T02:32:11.386ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24122025-01-15T02:32:11.387ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "118e7045-f440-41e6-b5f4-828bd44f7a95", "date": "Wed, 15 Jan 2025 02:32:11 GMT"} })
24132025-01-15T02:32:11.389ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
24142025-01-15T02:32:11.389ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 882.777471ms
24152025-01-15T02:32:11.389ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 882.787461ms iteration = 2 status = {}
24162025-01-15T02:32:12.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6818fb7d-fc23-41cc-940f-02f2ac11197f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24172025-01-15T02:32:12.083ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6818fb7d-fc23-41cc-940f-02f2ac11197f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24182025-01-15T02:32:12.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2293 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6818fb7d-fc23-41cc-940f-02f2ac11197f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24192025-01-15T02:32:13.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e6e34d5-cc89-4ad5-bd10-890f9ef40016 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24202025-01-15T02:32:13.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e6e34d5-cc89-4ad5-bd10-890f9ef40016 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24212025-01-15T02:32:13.106ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2171 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e6e34d5-cc89-4ad5-bd10-890f9ef40016 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24222025-01-15T02:32:14.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe201a33-88a8-4656-9d95-3e27d71af6d5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24232025-01-15T02:32:14.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe201a33-88a8-4656-9d95-3e27d71af6d5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24242025-01-15T02:32:14.130ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2734 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fe201a33-88a8-4656-9d95-3e27d71af6d5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24252025-01-15T02:32:15.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4f3f904-40f5-4af8-a960-428c7dbb64aa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24262025-01-15T02:32:15.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4f3f904-40f5-4af8-a960-428c7dbb64aa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24272025-01-15T02:32:15.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2981 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4f3f904-40f5-4af8-a960-428c7dbb64aa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24282025-01-15T02:32:15.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 495 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 4697f47c-a873-4d1f-a8f3-9f729cf7f84f response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
24292025-01-15T02:32:16.173ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 06038760-b7b4-4f4f-9c16-ab420d7a6502 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24302025-01-15T02:32:16.173ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 06038760-b7b4-4f4f-9c16-ab420d7a6502 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24312025-01-15T02:32:16.175ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 06038760-b7b4-4f4f-9c16-ab420d7a6502 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24322025-01-15T02:32:17.194ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5d4d31c-69a8-4c10-929c-cd1a7f7adb1b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24332025-01-15T02:32:17.195ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5d4d31c-69a8-4c10-929c-cd1a7f7adb1b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24342025-01-15T02:32:17.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5d4d31c-69a8-4c10-929c-cd1a7f7adb1b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24352025-01-15T02:32:18.216ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ec3b3df6-e8e2-4ace-ad61-5cb89e16c72b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24362025-01-15T02:32:18.216ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ec3b3df6-e8e2-4ace-ad61-5cb89e16c72b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24372025-01-15T02:32:18.218ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1896 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ec3b3df6-e8e2-4ace-ad61-5cb89e16c72b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24382025-01-15T02:32:19.237ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 01616dfd-fd85-4d1c-9ec1-653628aeee6e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24392025-01-15T02:32:19.237ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 01616dfd-fd85-4d1c-9ec1-653628aeee6e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24402025-01-15T02:32:19.239ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1804 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 01616dfd-fd85-4d1c-9ec1-653628aeee6e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24412025-01-15T02:32:20.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c26f4c33-a29a-4039-a78f-27afdd5f4632 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24422025-01-15T02:32:20.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c26f4c33-a29a-4039-a78f-27afdd5f4632 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24432025-01-15T02:32:20.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c26f4c33-a29a-4039-a78f-27afdd5f4632 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24442025-01-15T02:32:20.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
24452025-01-15T02:32:20.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
24462025-01-15T02:32:20.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24472025-01-15T02:32:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24482025-01-15T02:32:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24492025-01-15T02:32:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24502025-01-15T02:32:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24512025-01-15T02:32:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24522025-01-15T02:32:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24532025-01-15T02:32:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24542025-01-15T02:32:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24552025-01-15T02:32:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24562025-01-15T02:32:20.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24572025-01-15T02:32:20.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
24582025-01-15T02:32:20.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24592025-01-15T02:32:20.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24602025-01-15T02:32:20.528ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
24612025-01-15T02:32:20.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.728927ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
24622025-01-15T02:32:20.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24632025-01-15T02:32:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24642025-01-15T02:32:20.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.424069ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24652025-01-15T02:32:21.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a2652634-123e-4c29-9277-616c1af7034f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24662025-01-15T02:32:21.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a2652634-123e-4c29-9277-616c1af7034f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24672025-01-15T02:32:21.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2058 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a2652634-123e-4c29-9277-616c1af7034f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24682025-01-15T02:32:22.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ad06ebe-c297-4ad1-8138-889ff4b548f1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24692025-01-15T02:32:22.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ad06ebe-c297-4ad1-8138-889ff4b548f1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24702025-01-15T02:32:22.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1751 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ad06ebe-c297-4ad1-8138-889ff4b548f1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24712025-01-15T02:32:23.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8c4516ab-0b5f-43eb-89bb-eb47ff042bc5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24722025-01-15T02:32:23.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8c4516ab-0b5f-43eb-89bb-eb47ff042bc5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24732025-01-15T02:32:23.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1939 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8c4516ab-0b5f-43eb-89bb-eb47ff042bc5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24742025-01-15T02:32:24.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be348fdb-9f3f-41c5-b24a-cac2940af571 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24752025-01-15T02:32:24.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be348fdb-9f3f-41c5-b24a-cac2940af571 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24762025-01-15T02:32:24.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1784 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be348fdb-9f3f-41c5-b24a-cac2940af571 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24772025-01-15T02:32:25.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3f5e94c-fe52-4667-9140-105f1a6ce0e9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24782025-01-15T02:32:25.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3f5e94c-fe52-4667-9140-105f1a6ce0e9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24792025-01-15T02:32:25.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2316 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3f5e94c-fe52-4667-9140-105f1a6ce0e9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24802025-01-15T02:32:25.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 509 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = a5b6820c-e1c6-46cd-9b24-8c9543d0356f response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
24812025-01-15T02:32:26.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e382bb1-2981-4413-8253-a14cff2acba1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24822025-01-15T02:32:26.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e382bb1-2981-4413-8253-a14cff2acba1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24832025-01-15T02:32:26.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2295 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e382bb1-2981-4413-8253-a14cff2acba1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24842025-01-15T02:32:27.413ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5e8d8005-6698-49e2-9b63-324622227878 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24852025-01-15T02:32:27.414ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5e8d8005-6698-49e2-9b63-324622227878 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24862025-01-15T02:32:27.415ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1659 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5e8d8005-6698-49e2-9b63-324622227878 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24872025-01-15T02:32:28.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 19277795-c919-4d6a-ae42-006865a14ee8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24882025-01-15T02:32:28.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 19277795-c919-4d6a-ae42-006865a14ee8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24892025-01-15T02:32:28.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2325 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 19277795-c919-4d6a-ae42-006865a14ee8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24902025-01-15T02:32:29.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2507fdd6-cf5b-4d08-aa69-ca6f104b988d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24912025-01-15T02:32:29.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2507fdd6-cf5b-4d08-aa69-ca6f104b988d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24922025-01-15T02:32:29.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2093 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2507fdd6-cf5b-4d08-aa69-ca6f104b988d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24932025-01-15T02:32:30.478ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = af0343bb-07b8-4f92-a85e-5836a2477a20 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24942025-01-15T02:32:30.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = af0343bb-07b8-4f92-a85e-5836a2477a20 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24952025-01-15T02:32:30.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2448 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = af0343bb-07b8-4f92-a85e-5836a2477a20 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
24962025-01-15T02:32:30.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
24972025-01-15T02:32:30.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
24982025-01-15T02:32:30.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24992025-01-15T02:32:30.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25002025-01-15T02:32:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25012025-01-15T02:32:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25022025-01-15T02:32:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25032025-01-15T02:32:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25042025-01-15T02:32:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25052025-01-15T02:32:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25062025-01-15T02:32:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25072025-01-15T02:32:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25082025-01-15T02:32:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25092025-01-15T02:32:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
25102025-01-15T02:32:30.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25112025-01-15T02:32:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25122025-01-15T02:32:30.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
25132025-01-15T02:32:30.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.811401ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
25142025-01-15T02:32:30.535ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25152025-01-15T02:32:30.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25162025-01-15T02:32:30.537ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.478962ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25172025-01-15T02:32:31.500ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d183b074-8203-4940-bfed-649239ecd0d4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25182025-01-15T02:32:31.500ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d183b074-8203-4940-bfed-649239ecd0d4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25192025-01-15T02:32:31.501ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1642 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d183b074-8203-4940-bfed-649239ecd0d4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25202025-01-15T02:32:32.520ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32949b4b-4ca6-41d4-99d0-ee24885682a2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25212025-01-15T02:32:32.520ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32949b4b-4ca6-41d4-99d0-ee24885682a2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25222025-01-15T02:32:32.522ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2629 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32949b4b-4ca6-41d4-99d0-ee24885682a2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25232025-01-15T02:32:33.542ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a612e3a-66ce-43bb-9a22-9c37f3bb85a9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25242025-01-15T02:32:33.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a612e3a-66ce-43bb-9a22-9c37f3bb85a9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25252025-01-15T02:32:33.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2601 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a612e3a-66ce-43bb-9a22-9c37f3bb85a9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25262025-01-15T02:32:34.564ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 69293ebe-9577-4332-b075-1ef3b024f7c7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25272025-01-15T02:32:34.564ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 69293ebe-9577-4332-b075-1ef3b024f7c7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25282025-01-15T02:32:34.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1797 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 69293ebe-9577-4332-b075-1ef3b024f7c7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25292025-01-15T02:32:35.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 781c8598-a0e8-4657-8107-d0013b2e9c2a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25302025-01-15T02:32:35.586ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 781c8598-a0e8-4657-8107-d0013b2e9c2a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25312025-01-15T02:32:35.588ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2066 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 781c8598-a0e8-4657-8107-d0013b2e9c2a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25322025-01-15T02:32:35.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 682 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 8dfc315d-48bf-4899-8662-de060c7ca566 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
25332025-01-15T02:32:36.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc89116f-b0d3-4424-89cb-974b3531e65c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25342025-01-15T02:32:36.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc89116f-b0d3-4424-89cb-974b3531e65c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25352025-01-15T02:32:36.608ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1588 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc89116f-b0d3-4424-89cb-974b3531e65c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25362025-01-15T02:32:37.628ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 375ca672-17ef-4fdf-b8ce-3c444a714ed0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25372025-01-15T02:32:37.629ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 375ca672-17ef-4fdf-b8ce-3c444a714ed0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25382025-01-15T02:32:37.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 375ca672-17ef-4fdf-b8ce-3c444a714ed0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25392025-01-15T02:32:38.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2bc3bd01-15a1-4a24-a685-125f54177a8a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25402025-01-15T02:32:38.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2bc3bd01-15a1-4a24-a685-125f54177a8a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25412025-01-15T02:32:38.653ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2320 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2bc3bd01-15a1-4a24-a685-125f54177a8a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25422025-01-15T02:32:39.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a97bb26d-b6ad-4e35-a2e8-b3431b061404 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25432025-01-15T02:32:39.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a97bb26d-b6ad-4e35-a2e8-b3431b061404 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25442025-01-15T02:32:39.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2423 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a97bb26d-b6ad-4e35-a2e8-b3431b061404 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
25452025-01-15T02:32:40.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
25462025-01-15T02:32:40.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25472025-01-15T02:32:40.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
25482025-01-15T02:32:40.505ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
25492025-01-15T02:32:40.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
25502025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25512025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25522025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
25532025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
25542025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
25552025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25562025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
25572025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25582025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25592025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25602025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
25612025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25622025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
25632025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
25642025-01-15T02:32:40.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
25652025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25662025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25672025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
25682025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
25692025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
25702025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25712025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
25722025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25732025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25742025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
25752025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25762025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25772025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25782025-01-15T02:32:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25792025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25802025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25812025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25822025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25832025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25842025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25852025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25862025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25872025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
25882025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
25892025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25902025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
25912025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
25922025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
25932025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
25942025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25952025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
25962025-01-15T02:32:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25972025-01-15T02:32:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
25982025-01-15T02:32:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25992025-01-15T02:32:40.509ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26002025-01-15T02:32:40.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3647, tv_nsec: 344780968 } })
26012025-01-15T02:32:40.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3647, tv_nsec: 344795999 } })
26022025-01-15T02:32:40.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
26032025-01-15T02:32:40.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
26042025-01-15T02:32:40.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.323997ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
26052025-01-15T02:32:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26062025-01-15T02:32:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26072025-01-15T02:32:40.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26082025-01-15T02:32:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26092025-01-15T02:32:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26102025-01-15T02:32:40.513ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26112025-01-15T02:32:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26122025-01-15T02:32:40.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26132025-01-15T02:32:40.518ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26142025-01-15T02:32:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26152025-01-15T02:32:40.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26162025-01-15T02:32:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26172025-01-15T02:32:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
26182025-01-15T02:32:40.520ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26192025-01-15T02:32:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26202025-01-15T02:32:40.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26212025-01-15T02:32:40.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26222025-01-15T02:32:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26232025-01-15T02:32:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26242025-01-15T02:32:40.554ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26252025-01-15T02:32:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26262025-01-15T02:32:40.558ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26272025-01-15T02:32:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26282025-01-15T02:32:40.561ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26292025-01-15T02:32:40.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26302025-01-15T02:32:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26312025-01-15T02:32:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26322025-01-15T02:32:40.563ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26332025-01-15T02:32:40.563ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
26342025-01-15T02:32:40.563ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26352025-01-15T02:32:40.563ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26362025-01-15T02:32:40.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26372025-01-15T02:32:40.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26382025-01-15T02:32:40.565ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26392025-01-15T02:32:40.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26402025-01-15T02:32:40.566ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
26412025-01-15T02:32:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
26422025-01-15T02:32:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26432025-01-15T02:32:40.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26442025-01-15T02:32:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26452025-01-15T02:32:40.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.007536ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
26462025-01-15T02:32:40.568ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26472025-01-15T02:32:40.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26482025-01-15T02:32:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26492025-01-15T02:32:40.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26502025-01-15T02:32:40.571ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3647, tv_nsec: 406763341 } })
26512025-01-15T02:32:40.571ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
26522025-01-15T02:32:40.572ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26532025-01-15T02:32:40.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26542025-01-15T02:32:40.572ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26552025-01-15T02:32:40.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26562025-01-15T02:32:40.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26572025-01-15T02:32:40.574ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26582025-01-15T02:32:40.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26592025-01-15T02:32:40.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26602025-01-15T02:32:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.926776ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
26612025-01-15T02:32:40.575ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
26622025-01-15T02:32:40.575ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 68.553849ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
26632025-01-15T02:32:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
26642025-01-15T02:32:40.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26652025-01-15T02:32:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26662025-01-15T02:32:40.577ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26672025-01-15T02:32:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
26682025-01-15T02:32:40.580ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26692025-01-15T02:32:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26702025-01-15T02:32:40.585ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26712025-01-15T02:32:40.587ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26722025-01-15T02:32:40.587ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26732025-01-15T02:32:40.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26742025-01-15T02:32:40.590ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
26752025-01-15T02:32:40.590ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.75595ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
26762025-01-15T02:32:40.594ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26772025-01-15T02:32:40.594ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.287913ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26782025-01-15T02:32:40.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26792025-01-15T02:32:40.601ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 93.716159ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26802025-01-15T02:32:40.602ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 95.458746ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26812025-01-15T02:32:40.607ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26822025-01-15T02:32:40.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26832025-01-15T02:32:40.609ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26842025-01-15T02:32:40.609ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
26852025-01-15T02:32:40.609ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26862025-01-15T02:32:40.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26872025-01-15T02:32:40.637ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26882025-01-15T02:32:40.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26892025-01-15T02:32:40.696ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
26902025-01-15T02:32:40.696ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26912025-01-15T02:32:40.696ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a49271c4-9314-4989-bf86-d52f26184e52", "content-length": "26", "date": "Wed, 15 Jan 2025 02:32:39 GMT"} })
26922025-01-15T02:32:40.696ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26932025-01-15T02:32:40.697ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
26942025-01-15T02:32:40.697ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26952025-01-15T02:32:40.697ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "be9146a0-0c7c-42b0-9ccf-e25119a3fea1", "content-length": "26", "date": "Wed, 15 Jan 2025 02:32:39 GMT"} })
26962025-01-15T02:32:40.697ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26972025-01-15T02:32:40.698ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
26982025-01-15T02:32:40.698ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26992025-01-15T02:32:40.698ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "307b837c-c35f-483a-8f7e-351dd656222c", "content-length": "26", "date": "Wed, 15 Jan 2025 02:32:39 GMT"} })
27002025-01-15T02:32:40.698ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27012025-01-15T02:32:40.700ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27022025-01-15T02:32:40.701ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27032025-01-15T02:32:40.701ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
27042025-01-15T02:32:40.702ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
27052025-01-15T02:32:40.702ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27062025-01-15T02:32:40.710ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27072025-01-15T02:32:40.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27082025-01-15T02:32:40.741ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27092025-01-15T02:32:40.741ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
27102025-01-15T02:32:40.741ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 234.95756ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
27112025-01-15T02:32:40.747ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
27122025-01-15T02:32:40.747ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 240.833949ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
27132025-01-15T02:32:40.749ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 241.99051ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
27142025-01-15T02:32:40.752ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27152025-01-15T02:32:40.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
27162025-01-15T02:32:40.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27172025-01-15T02:32:40.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27182025-01-15T02:32:40.758ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 252.126364ms iteration = 12 status = 0
27192025-01-15T02:32:40.800ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
27202025-01-15T02:32:40.801ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f7202f0-c432-48e1-89c9-45d2584e6ad3", "content-length": "1", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27212025-01-15T02:32:40.803ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 297.297943ms iteration = 12 status = {}
27222025-01-15T02:32:40.803ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27232025-01-15T02:32:40.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27242025-01-15T02:32:40.806ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 300.617872ms iteration = 12 status = null
27252025-01-15T02:32:40.810ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c33785f-1afc-45d0-88ca-7c8b32e85950", "content-length": "372", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27262025-01-15T02:32:40.810ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27272025-01-15T02:32:40.810ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ceb6020-04a5-48c0-847a-b6af4a1f1b71 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
27282025-01-15T02:32:40.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27292025-01-15T02:32:40.810ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ceb6020-04a5-48c0-847a-b6af4a1f1b71 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
27302025-01-15T02:32:40.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2950 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ceb6020-04a5-48c0-847a-b6af4a1f1b71 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
27312025-01-15T02:32:40.814ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 307.678993ms iteration = 14 status = {"physical_disks_added":0}
27322025-01-15T02:32:40.816ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27332025-01-15T02:32:40.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27342025-01-15T02:32:40.850ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27352025-01-15T02:32:40.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
27362025-01-15T02:32:40.851ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27372025-01-15T02:32:40.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27382025-01-15T02:32:40.853ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
27392025-01-15T02:32:40.853ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
27402025-01-15T02:32:40.853ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27412025-01-15T02:32:40.853ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "606c6a26-fc81-4263-8cb9-ee173ddb2d14", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27422025-01-15T02:32:40.853ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27432025-01-15T02:32:40.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27442025-01-15T02:32:40.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
27452025-01-15T02:32:40.855ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27462025-01-15T02:32:40.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27472025-01-15T02:32:40.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27482025-01-15T02:32:40.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27492025-01-15T02:32:40.858ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27502025-01-15T02:32:40.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27512025-01-15T02:32:40.862ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
27522025-01-15T02:32:40.862ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27532025-01-15T02:32:40.862ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c114055c-df0f-4c91-a5b5-33248a61779e", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27542025-01-15T02:32:40.864ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 358.328689ms iteration = 12 status = {}
27552025-01-15T02:32:40.866ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27562025-01-15T02:32:40.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27572025-01-15T02:32:40.868ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
27582025-01-15T02:32:40.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27592025-01-15T02:32:40.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27602025-01-15T02:32:40.877ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
27612025-01-15T02:32:40.877ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbb8b8f1-b7c4-441b-a67a-946fa96f8500", "content-length": "134", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27622025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
27632025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
27642025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
27652025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27662025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27672025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27682025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27692025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27702025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27712025-01-15T02:32:40.877ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27722025-01-15T02:32:40.878ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
27732025-01-15T02:32:40.878ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27742025-01-15T02:32:40.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27752025-01-15T02:32:40.889ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
27762025-01-15T02:32:40.890ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "baa468a3-3fcf-457e-814d-4e0511347d7d", "content-length": "691", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27772025-01-15T02:32:40.890ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
27782025-01-15T02:32:40.890ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
27792025-01-15T02:32:40.890ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
27802025-01-15T02:32:40.891ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
27812025-01-15T02:32:40.891ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4daece6-256c-430f-ba7e-8e28994e1fde", "content-length": "2", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27822025-01-15T02:32:40.891ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
27832025-01-15T02:32:40.891ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccc7b81a-2fd2-4961-83b0-c01894a51f8b", "content-length": "52", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27842025-01-15T02:32:40.891ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
27852025-01-15T02:32:40.891ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27862025-01-15T02:32:40.892ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 385.76728ms iteration = 12 status = {}
27872025-01-15T02:32:40.988ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3b8a59f8-ef80-4eab-8a83-cdc76c9f65ae", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27882025-01-15T02:32:40.988ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
27892025-01-15T02:32:40.988ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27902025-01-15T02:32:40.988ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d061a47-044a-4071-96af-53f8f4573925", "content-length": "682", "date": "Wed, 15 Jan 2025 02:32:40 GMT"} })
27912025-01-15T02:32:40.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27922025-01-15T02:32:40.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27932025-01-15T02:32:40.992ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27942025-01-15T02:32:40.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27952025-01-15T02:32:40.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27962025-01-15T02:32:40.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27972025-01-15T02:32:40.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27982025-01-15T02:32:40.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27992025-01-15T02:32:41.014ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 507.620117ms iteration = 12 status = {}
28002025-01-15T02:32:41.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 41102591-3b85-47c3-8d99-49f999e8353b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28012025-01-15T02:32:41.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 41102591-3b85-47c3-8d99-49f999e8353b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28022025-01-15T02:32:41.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1987 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 41102591-3b85-47c3-8d99-49f999e8353b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28032025-01-15T02:32:42.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7adf0e4a-332b-4849-83eb-5262c6a25c02 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28042025-01-15T02:32:42.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7adf0e4a-332b-4849-83eb-5262c6a25c02 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28052025-01-15T02:32:42.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2003 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7adf0e4a-332b-4849-83eb-5262c6a25c02 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28062025-01-15T02:32:43.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = faaf9f52-61b7-4aac-ba16-17afeeff192f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28072025-01-15T02:32:43.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = faaf9f52-61b7-4aac-ba16-17afeeff192f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28082025-01-15T02:32:43.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2434 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = faaf9f52-61b7-4aac-ba16-17afeeff192f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28092025-01-15T02:32:44.900ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 11816643-eec9-4ab3-aaa3-630152b09c67 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28102025-01-15T02:32:44.900ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 11816643-eec9-4ab3-aaa3-630152b09c67 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28112025-01-15T02:32:44.902ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2564 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 11816643-eec9-4ab3-aaa3-630152b09c67 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28122025-01-15T02:32:45.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 493 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = c622aaca-e386-4e2d-8483-0ee2c72f55a7 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
28132025-01-15T02:32:45.922ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6b2ac5f6-1927-4b6d-926a-10ccfa1b1add roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28142025-01-15T02:32:45.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6b2ac5f6-1927-4b6d-926a-10ccfa1b1add resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28152025-01-15T02:32:45.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2267 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6b2ac5f6-1927-4b6d-926a-10ccfa1b1add response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28162025-01-15T02:32:46.944ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 36fd41fa-8820-4c9c-9be6-bdd64a19bfe0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28172025-01-15T02:32:46.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 36fd41fa-8820-4c9c-9be6-bdd64a19bfe0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28182025-01-15T02:32:46.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2008 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 36fd41fa-8820-4c9c-9be6-bdd64a19bfe0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28192025-01-15T02:32:47.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 20ca98ce-81a6-4380-a9c3-090ca4a7d0f4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28202025-01-15T02:32:47.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 20ca98ce-81a6-4380-a9c3-090ca4a7d0f4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28212025-01-15T02:32:47.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2009 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 20ca98ce-81a6-4380-a9c3-090ca4a7d0f4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28222025-01-15T02:32:48.987ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e6273f56-6d28-47ed-8883-bf47f15a18bc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28232025-01-15T02:32:48.988ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e6273f56-6d28-47ed-8883-bf47f15a18bc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28242025-01-15T02:32:48.989ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2224 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e6273f56-6d28-47ed-8883-bf47f15a18bc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28252025-01-15T02:32:50.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 76e5c700-eaa5-4919-90b5-a97a1a55602b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28262025-01-15T02:32:50.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 76e5c700-eaa5-4919-90b5-a97a1a55602b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28272025-01-15T02:32:50.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2183 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 76e5c700-eaa5-4919-90b5-a97a1a55602b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28282025-01-15T02:32:50.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
28292025-01-15T02:32:50.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
28302025-01-15T02:32:50.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28312025-01-15T02:32:50.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28322025-01-15T02:32:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28332025-01-15T02:32:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28342025-01-15T02:32:50.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28352025-01-15T02:32:50.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28362025-01-15T02:32:50.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28372025-01-15T02:32:50.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.918877ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28382025-01-15T02:32:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28392025-01-15T02:32:50.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28402025-01-15T02:32:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28412025-01-15T02:32:50.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28422025-01-15T02:32:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28432025-01-15T02:32:50.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28442025-01-15T02:32:50.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
28452025-01-15T02:32:50.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28462025-01-15T02:32:50.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28472025-01-15T02:32:50.555ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
28482025-01-15T02:32:50.555ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.507706ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
28492025-01-15T02:32:51.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4910595-3c77-4421-9ff9-1ad847a99428 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28502025-01-15T02:32:51.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4910595-3c77-4421-9ff9-1ad847a99428 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28512025-01-15T02:32:51.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1856 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4910595-3c77-4421-9ff9-1ad847a99428 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28522025-01-15T02:32:52.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ae9473d-7b42-4733-b4db-77e05caca0b0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28532025-01-15T02:32:52.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ae9473d-7b42-4733-b4db-77e05caca0b0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28542025-01-15T02:32:52.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2174 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ae9473d-7b42-4733-b4db-77e05caca0b0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28552025-01-15T02:32:53.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 07b47fe2-e244-42c5-a449-49559c94d54d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28562025-01-15T02:32:53.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 07b47fe2-e244-42c5-a449-49559c94d54d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28572025-01-15T02:32:53.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2743 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 07b47fe2-e244-42c5-a449-49559c94d54d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28582025-01-15T02:32:54.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 498105d5-82d2-46b3-8ea9-7aec6084fdd2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28592025-01-15T02:32:54.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 498105d5-82d2-46b3-8ea9-7aec6084fdd2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28602025-01-15T02:32:54.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1966 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 498105d5-82d2-46b3-8ea9-7aec6084fdd2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28612025-01-15T02:32:55.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c079b4c4-70bb-42f0-8573-316c5ba885ed roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28622025-01-15T02:32:55.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c079b4c4-70bb-42f0-8573-316c5ba885ed resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28632025-01-15T02:32:55.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2433 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c079b4c4-70bb-42f0-8573-316c5ba885ed response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28642025-01-15T02:32:55.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 406 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 6c984ca0-1cf3-4ce7-9f21-8fd0a14408ff response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
28652025-01-15T02:32:56.140ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84791907-c377-46f8-9bf0-082c1dbe8613 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28662025-01-15T02:32:56.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84791907-c377-46f8-9bf0-082c1dbe8613 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28672025-01-15T02:32:56.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2033 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84791907-c377-46f8-9bf0-082c1dbe8613 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28682025-01-15T02:32:57.162ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 200b7b4a-d39a-4694-9b0b-e7811c8406c9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28692025-01-15T02:32:57.162ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 200b7b4a-d39a-4694-9b0b-e7811c8406c9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28702025-01-15T02:32:57.164ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2039 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 200b7b4a-d39a-4694-9b0b-e7811c8406c9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28712025-01-15T02:32:58.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1a7b571d-f584-4c0a-b63c-244e7a5aee0a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28722025-01-15T02:32:58.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1a7b571d-f584-4c0a-b63c-244e7a5aee0a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28732025-01-15T02:32:58.187ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1a7b571d-f584-4c0a-b63c-244e7a5aee0a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28742025-01-15T02:32:59.207ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84f24165-7f38-462a-9ae0-78bc1dae053a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28752025-01-15T02:32:59.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84f24165-7f38-462a-9ae0-78bc1dae053a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28762025-01-15T02:32:59.209ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84f24165-7f38-462a-9ae0-78bc1dae053a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28772025-01-15T02:33:00.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = db60f0f0-5808-4f3e-9efd-96f24fde60fa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28782025-01-15T02:33:00.230ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = db60f0f0-5808-4f3e-9efd-96f24fde60fa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28792025-01-15T02:33:00.232ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2771 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = db60f0f0-5808-4f3e-9efd-96f24fde60fa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
28802025-01-15T02:33:00.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
28812025-01-15T02:33:00.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
28822025-01-15T02:33:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28832025-01-15T02:33:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28842025-01-15T02:33:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28852025-01-15T02:33:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28862025-01-15T02:33:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28872025-01-15T02:33:00.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28882025-01-15T02:33:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28892025-01-15T02:33:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28902025-01-15T02:33:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28912025-01-15T02:33:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28922025-01-15T02:33:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28932025-01-15T02:33:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28942025-01-15T02:33:00.538ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.23401ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28952025-01-15T02:33:00.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28962025-01-15T02:33:00.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
28972025-01-15T02:33:00.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28982025-01-15T02:33:00.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28992025-01-15T02:33:00.561ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
29002025-01-15T02:33:00.561ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.737587ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
29012025-01-15T02:33:01.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 262561d6-9784-4a16-80da-007b68568545 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29022025-01-15T02:33:01.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 262561d6-9784-4a16-80da-007b68568545 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29032025-01-15T02:33:01.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2381 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 262561d6-9784-4a16-80da-007b68568545 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29042025-01-15T02:33:02.273ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 71cf7dc4-37d8-4745-9e55-298f08acad92 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29052025-01-15T02:33:02.273ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 71cf7dc4-37d8-4745-9e55-298f08acad92 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29062025-01-15T02:33:02.275ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2063 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 71cf7dc4-37d8-4745-9e55-298f08acad92 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29072025-01-15T02:33:03.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 111e29d6-739b-4af8-ae12-8b320beac158 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29082025-01-15T02:33:03.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 111e29d6-739b-4af8-ae12-8b320beac158 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29092025-01-15T02:33:03.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1872 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 111e29d6-739b-4af8-ae12-8b320beac158 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29102025-01-15T02:33:04.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2d408552-b605-45a5-b655-60e6b393e494 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29112025-01-15T02:33:04.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2d408552-b605-45a5-b655-60e6b393e494 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29122025-01-15T02:33:04.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2202 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2d408552-b605-45a5-b655-60e6b393e494 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29132025-01-15T02:33:05.339ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a1e9c38b-3e6a-4bcb-9cc2-5c6397aa20b8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29142025-01-15T02:33:05.339ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a1e9c38b-3e6a-4bcb-9cc2-5c6397aa20b8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29152025-01-15T02:33:05.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1988 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a1e9c38b-3e6a-4bcb-9cc2-5c6397aa20b8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29162025-01-15T02:33:05.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 764 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 823c768c-e897-445d-9c12-220471c2c58b response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
29172025-01-15T02:33:06.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ab661fc-8b88-4c09-a2b0-a4871e31c60e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29182025-01-15T02:33:06.361ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ab661fc-8b88-4c09-a2b0-a4871e31c60e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29192025-01-15T02:33:06.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2463 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5ab661fc-8b88-4c09-a2b0-a4871e31c60e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29202025-01-15T02:33:07.382ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b80b6094-f6f3-47fd-ad21-4afe8e3b04a2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29212025-01-15T02:33:07.383ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b80b6094-f6f3-47fd-ad21-4afe8e3b04a2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29222025-01-15T02:33:07.384ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1972 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b80b6094-f6f3-47fd-ad21-4afe8e3b04a2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29232025-01-15T02:33:08.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 54dc1db1-1b4b-4862-bb5b-046e12527972 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29242025-01-15T02:33:08.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 54dc1db1-1b4b-4862-bb5b-046e12527972 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29252025-01-15T02:33:08.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2208 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 54dc1db1-1b4b-4862-bb5b-046e12527972 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29262025-01-15T02:33:09.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3e631cbb-a40b-48c7-8da3-1788a254ac21 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29272025-01-15T02:33:09.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3e631cbb-a40b-48c7-8da3-1788a254ac21 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29282025-01-15T02:33:09.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8538 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3e631cbb-a40b-48c7-8da3-1788a254ac21 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29292025-01-15T02:33:10.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0285c2ed-beb5-45c3-879a-077fcff62443 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29302025-01-15T02:33:10.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0285c2ed-beb5-45c3-879a-077fcff62443 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29312025-01-15T02:33:10.462ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1988 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0285c2ed-beb5-45c3-879a-077fcff62443 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
29322025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
29332025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
29342025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
29352025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
29362025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
29372025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
29382025-01-15T02:33:10.505ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:23:10.505064199 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
29392025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
29402025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
29412025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29422025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
29432025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29442025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
29452025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29462025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
29472025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
29482025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
29492025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
29502025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29512025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29522025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29532025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29542025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
29552025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29562025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
29572025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29582025-01-15T02:33:10.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3677, tv_nsec: 341382267 } })
29592025-01-15T02:33:10.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3677, tv_nsec: 341383967 } })
29602025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
29612025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
29622025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
29632025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
29642025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29652025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29662025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29672025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
29682025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29692025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
29702025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
29712025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
29722025-01-15T02:33:10.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
29732025-01-15T02:33:10.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29742025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
29752025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29762025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
29772025-01-15T02:33:10.506ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
29782025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
29792025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.23\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
29802025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
29812025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
29822025-01-15T02:33:10.506ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
29832025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
29842025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
29852025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
29862025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29872025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29882025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
29892025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29902025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
29912025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29922025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
29932025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29942025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29952025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29962025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29972025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29982025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
29992025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30002025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30012025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30022025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30032025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
30042025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30052025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30062025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30072025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
30082025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30092025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.566892ms iteration = 7 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
30102025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30112025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
30122025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.539542ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
30132025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30142025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30152025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30162025-01-15T02:33:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30172025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30182025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30192025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30202025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
30212025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30222025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
30232025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
30242025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
30252025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
30262025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
30272025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
30282025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
30292025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
30302025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
30312025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30322025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30332025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
30342025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
30352025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
30362025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30372025-01-15T02:33:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
30382025-01-15T02:33:10.509ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30392025-01-15T02:33:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30402025-01-15T02:33:10.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30412025-01-15T02:33:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30422025-01-15T02:33:10.511ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30432025-01-15T02:33:10.512ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:23:10.505064199 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
30442025-01-15T02:33:10.512ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.368529ms iteration = 7 status = {"expiration":"2025-01-15T02:23:10.505064199Z","pruned":[]}
30452025-01-15T02:33:10.512ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.39592ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
30462025-01-15T02:33:10.513ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30472025-01-15T02:33:10.513ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30482025-01-15T02:33:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30492025-01-15T02:33:10.513ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3677, tv_nsec: 349218318 } })
30502025-01-15T02:33:10.513ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
30512025-01-15T02:33:10.514ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3677, tv_nsec: 349670110 } })
30522025-01-15T02:33:10.514ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
30532025-01-15T02:33:10.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30542025-01-15T02:33:10.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30552025-01-15T02:33:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30562025-01-15T02:33:10.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30572025-01-15T02:33:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30582025-01-15T02:33:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30592025-01-15T02:33:10.519ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30602025-01-15T02:33:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30612025-01-15T02:33:10.523ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30622025-01-15T02:33:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
30632025-01-15T02:33:10.526ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
30642025-01-15T02:33:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
30652025-01-15T02:33:10.531ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 24.168822ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
30662025-01-15T02:33:10.532ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30672025-01-15T02:33:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
30682025-01-15T02:33:10.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30692025-01-15T02:33:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30702025-01-15T02:33:10.556ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30712025-01-15T02:33:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30722025-01-15T02:33:10.559ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30732025-01-15T02:33:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30742025-01-15T02:33:10.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30752025-01-15T02:33:10.568ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30762025-01-15T02:33:10.568ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
30772025-01-15T02:33:10.568ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30782025-01-15T02:33:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30792025-01-15T02:33:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30802025-01-15T02:33:10.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30812025-01-15T02:33:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30822025-01-15T02:33:10.571ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30832025-01-15T02:33:10.572ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30842025-01-15T02:33:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30852025-01-15T02:33:10.573ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30862025-01-15T02:33:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30872025-01-15T02:33:10.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30882025-01-15T02:33:10.574ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30892025-01-15T02:33:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30902025-01-15T02:33:10.575ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30912025-01-15T02:33:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30922025-01-15T02:33:10.575ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30932025-01-15T02:33:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30942025-01-15T02:33:10.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30952025-01-15T02:33:10.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30962025-01-15T02:33:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30972025-01-15T02:33:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30982025-01-15T02:33:10.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30992025-01-15T02:33:10.577ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31002025-01-15T02:33:10.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31012025-01-15T02:33:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31022025-01-15T02:33:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31032025-01-15T02:33:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31042025-01-15T02:33:10.579ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31052025-01-15T02:33:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31062025-01-15T02:33:10.581ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31072025-01-15T02:33:10.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31082025-01-15T02:33:10.581ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31092025-01-15T02:33:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31102025-01-15T02:33:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31112025-01-15T02:33:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31122025-01-15T02:33:10.583ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31132025-01-15T02:33:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31142025-01-15T02:33:10.583ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31152025-01-15T02:33:10.585ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31162025-01-15T02:33:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31172025-01-15T02:33:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31182025-01-15T02:33:10.586ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31192025-01-15T02:33:10.586ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
31202025-01-15T02:33:10.586ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
31212025-01-15T02:33:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 78.688873ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
31222025-01-15T02:33:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31232025-01-15T02:33:10.586ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31242025-01-15T02:33:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31252025-01-15T02:33:10.588ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31262025-01-15T02:33:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31272025-01-15T02:33:10.589ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 81.54754ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
31282025-01-15T02:33:10.591ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3677, tv_nsec: 426962226 } })
31292025-01-15T02:33:10.591ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31302025-01-15T02:33:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31312025-01-15T02:33:10.593ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
31322025-01-15T02:33:10.596ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31332025-01-15T02:33:10.597ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31342025-01-15T02:33:10.599ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31352025-01-15T02:33:10.599ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31362025-01-15T02:33:10.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31372025-01-15T02:33:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
31382025-01-15T02:33:10.601ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31392025-01-15T02:33:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31402025-01-15T02:33:10.602ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31412025-01-15T02:33:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31422025-01-15T02:33:10.604ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
31432025-01-15T02:33:10.604ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 98.718943ms iteration = 4 status = {"nsuccess":5}
31442025-01-15T02:33:10.606ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31452025-01-15T02:33:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31462025-01-15T02:33:10.609ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31472025-01-15T02:33:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31482025-01-15T02:33:10.611ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 104.306554ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
31492025-01-15T02:33:10.616ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
31502025-01-15T02:33:10.617ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31512025-01-15T02:33:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31522025-01-15T02:33:10.626ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
31532025-01-15T02:33:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
31542025-01-15T02:33:10.628ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31552025-01-15T02:33:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31562025-01-15T02:33:10.630ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31572025-01-15T02:33:10.630ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31582025-01-15T02:33:10.632ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
31592025-01-15T02:33:10.632ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 126.709369ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
31602025-01-15T02:33:10.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31612025-01-15T02:33:10.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31622025-01-15T02:33:10.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31632025-01-15T02:33:10.634ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
31642025-01-15T02:33:10.634ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
31652025-01-15T02:33:10.634ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 128.200599ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
31662025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31672025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
31682025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
31692025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
31702025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31712025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31722025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31732025-01-15T02:33:10.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31742025-01-15T02:33:10.636ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31752025-01-15T02:33:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31762025-01-15T02:33:10.636ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31772025-01-15T02:33:10.637ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 131.041566ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
31782025-01-15T02:33:10.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31792025-01-15T02:33:10.639ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31802025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
31812025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31822025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31832025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31842025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31852025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31862025-01-15T02:33:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31872025-01-15T02:33:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31882025-01-15T02:33:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31892025-01-15T02:33:10.640ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31902025-01-15T02:33:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31912025-01-15T02:33:10.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-01-15T02:33:10.643ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31932025-01-15T02:33:10.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31942025-01-15T02:33:10.643ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 137.453189ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31952025-01-15T02:33:10.645ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31962025-01-15T02:33:10.645ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
31972025-01-15T02:33:10.645ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31982025-01-15T02:33:10.645ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2171fc1-5ca2-453b-ba16-e105648eecf8", "content-length": "26", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
31992025-01-15T02:33:10.645ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32002025-01-15T02:33:10.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
32012025-01-15T02:33:10.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32022025-01-15T02:33:10.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32032025-01-15T02:33:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32042025-01-15T02:33:10.647ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32052025-01-15T02:33:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32062025-01-15T02:33:10.648ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32072025-01-15T02:33:10.649ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 143.41624ms iteration = 13 status = 0
32082025-01-15T02:33:10.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32092025-01-15T02:33:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
32102025-01-15T02:33:10.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32112025-01-15T02:33:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32122025-01-15T02:33:10.656ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32132025-01-15T02:33:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32142025-01-15T02:33:10.658ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32152025-01-15T02:33:10.660ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
32162025-01-15T02:33:10.660ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32172025-01-15T02:33:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32182025-01-15T02:33:10.666ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 158.712113ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
32192025-01-15T02:33:10.670ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
32202025-01-15T02:33:10.670ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 164.352185ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
32212025-01-15T02:33:10.677ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
32222025-01-15T02:33:10.677ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 171.373485ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
32232025-01-15T02:33:10.683ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
32242025-01-15T02:33:10.683ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
32252025-01-15T02:33:10.683ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
32262025-01-15T02:33:10.683ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32272025-01-15T02:33:10.683ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 176.350659ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
32282025-01-15T02:33:10.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32292025-01-15T02:33:10.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32302025-01-15T02:33:10.687ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
32312025-01-15T02:33:10.687ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 181.297692ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
32322025-01-15T02:33:10.688ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32332025-01-15T02:33:10.688ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
32342025-01-15T02:33:10.688ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 182.669549ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
32352025-01-15T02:33:10.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32362025-01-15T02:33:10.689ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32372025-01-15T02:33:10.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32382025-01-15T02:33:10.690ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
32392025-01-15T02:33:10.690ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
32402025-01-15T02:33:10.690ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32412025-01-15T02:33:10.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32422025-01-15T02:33:10.709ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
32432025-01-15T02:33:10.709ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 204.19138ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
32442025-01-15T02:33:10.710ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
32452025-01-15T02:33:10.711ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32462025-01-15T02:33:10.711ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "388b9043-7c76-490c-b548-e511dd32b54d", "content-length": "26", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
32472025-01-15T02:33:10.711ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32482025-01-15T02:33:10.714ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32492025-01-15T02:33:10.730ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
32502025-01-15T02:33:10.730ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
32512025-01-15T02:33:10.730ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32522025-01-15T02:33:10.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32532025-01-15T02:33:10.748ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 242.921596ms iteration = 15 status = {"physical_disks_added":0}
32542025-01-15T02:33:10.793ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
32552025-01-15T02:33:10.844ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
32562025-01-15T02:33:10.844ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f62aec1f-21b6-488f-87c2-066d50cba404", "content-length": "1", "date": "Wed, 15 Jan 2025 02:33:09 GMT"} })
32572025-01-15T02:33:10.846ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32582025-01-15T02:33:10.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32592025-01-15T02:33:10.862ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
32602025-01-15T02:33:10.862ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "9888ca6e-ecbb-412d-a709-51d2b615d56f", "date": "Wed, 15 Jan 2025 02:33:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32612025-01-15T02:33:10.862ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32622025-01-15T02:33:10.864ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d4d901ba-8f98-48d8-b8fc-847894cd3b6f", "date": "Wed, 15 Jan 2025 02:33:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32632025-01-15T02:33:10.864ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32642025-01-15T02:33:10.866ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32652025-01-15T02:33:10.866ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 361.416393ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
32662025-01-15T02:33:10.878ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
32672025-01-15T02:33:10.878ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32682025-01-15T02:33:10.878ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f2b11b6-426d-46bc-8d1d-de8d60d5898b", "content-length": "26", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
32692025-01-15T02:33:10.878ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32702025-01-15T02:33:10.880ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
32712025-01-15T02:33:10.896ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 391.263168ms iteration = 13 status = {}
32722025-01-15T02:33:10.933ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32732025-01-15T02:33:10.933ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32742025-01-15T02:33:10.933ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f25931ff-e372-48ee-a058-65c5017fb3fc", "content-length": "372", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
32752025-01-15T02:33:10.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32762025-01-15T02:33:10.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32772025-01-15T02:33:10.934ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32782025-01-15T02:33:10.935ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32792025-01-15T02:33:10.936ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 431.551265ms iteration = 13 status = null
32802025-01-15T02:33:10.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32812025-01-15T02:33:10.938ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32822025-01-15T02:33:10.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32832025-01-15T02:33:10.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32842025-01-15T02:33:10.938ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32852025-01-15T02:33:10.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32862025-01-15T02:33:10.939ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
32872025-01-15T02:33:10.939ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "3ac6fcfd-c5e5-4a5b-970e-edcea3ef8378", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32882025-01-15T02:33:10.939ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32892025-01-15T02:33:10.940ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8464ba6f-e92a-437e-805b-22b05cb8ee91", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32902025-01-15T02:33:10.940ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32912025-01-15T02:33:10.942ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "772c685c-786d-42f4-8298-7047a8bb8b3f", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32922025-01-15T02:33:10.942ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32932025-01-15T02:33:10.943ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32942025-01-15T02:33:10.943ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "72c4913f-adb1-4256-b07e-bd7325250d84", "content-length": "691", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
32952025-01-15T02:33:10.943ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
32962025-01-15T02:33:10.943ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
32972025-01-15T02:33:10.943ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
32982025-01-15T02:33:10.944ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
32992025-01-15T02:33:10.944ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 439.404537ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
33002025-01-15T02:33:10.944ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 438.946495ms iteration = 13 status = {}
33012025-01-15T02:33:10.945ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
33022025-01-15T02:33:10.945ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
33032025-01-15T02:33:10.945ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
33042025-01-15T02:33:10.945ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 440.476035ms iteration = 7 status = {"generation":2}
33052025-01-15T02:33:10.946ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
33062025-01-15T02:33:10.946ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
33072025-01-15T02:33:10.946ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
33082025-01-15T02:33:10.946ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 441.300348ms iteration = 7 status = {"generation":1}
33092025-01-15T02:33:10.975ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
33102025-01-15T02:33:10.975ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33112025-01-15T02:33:10.975ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0cba6aa0-0aa2-41a7-8acb-185ee3470938", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33122025-01-15T02:33:10.975ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33132025-01-15T02:33:10.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33142025-01-15T02:33:10.977ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33152025-01-15T02:33:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33162025-01-15T02:33:10.980ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33172025-01-15T02:33:10.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33182025-01-15T02:33:10.984ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
33192025-01-15T02:33:10.984ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33202025-01-15T02:33:10.984ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c0e0b67f-e24c-4a15-95dc-09864a836135", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33212025-01-15T02:33:10.986ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 480.185037ms iteration = 13 status = {}
33222025-01-15T02:33:10.986ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
33232025-01-15T02:33:10.986ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
33242025-01-15T02:33:11.048ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33252025-01-15T02:33:11.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33262025-01-15T02:33:11.049ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
33272025-01-15T02:33:11.049ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:41083
33282025-01-15T02:33:11.049ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:41083 req_id = 5791e115-429e-4861-b505-a7d23d6bb42b roles = RoleSet { roles: {} } uri = /metrics/producers
33292025-01-15T02:33:11.049ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:41083 req_id = 5791e115-429e-4861-b505-a7d23d6bb42b resource = Database result = Ok(()) uri = /metrics/producers
33302025-01-15T02:33:11.050ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
33312025-01-15T02:33:11.050ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33322025-01-15T02:33:11.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33332025-01-15T02:33:11.060ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
33342025-01-15T02:33:11.060ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "10997c24-ce48-4164-b607-1e56c6db32ab", "content-length": "134", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33352025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33362025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
33372025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
33382025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33392025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33402025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33412025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33422025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33432025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33442025-01-15T02:33:11.060ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33452025-01-15T02:33:11.061ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
33462025-01-15T02:33:11.061ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33472025-01-15T02:33:11.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33482025-01-15T02:33:11.063ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
33492025-01-15T02:33:11.063ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ce842e6-4ff5-4466-8783-2fe70f76b580", "content-length": "2", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33502025-01-15T02:33:11.063ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
33512025-01-15T02:33:11.063ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e9217f0-674f-4e89-a70f-88bfb71dfda9", "content-length": "52", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33522025-01-15T02:33:11.063ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
33532025-01-15T02:33:11.063ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33542025-01-15T02:33:11.084ZINFOnexus: assigned collector to new producer collector_id = f71d33aa-7320-48b9-81d2-899a100e78cd file = nexus/src/app/oximeter.rs:76 producer_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
33552025-01-15T02:33:11.084ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35394 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:41083 req_id = 5791e115-429e-4861-b505-a7d23d6bb42b response_code = 201 uri = /metrics/producers
33562025-01-15T02:33:11.084ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "5791e115-429e-4861-b505-a7d23d6bb42b", "content-length": "41", "date": "Wed, 15 Jan 2025 02:33:11 GMT"} })
33572025-01-15T02:33:11.086ZDEBGnexus (producer-registration-task): registered with nexus successfully
33582025-01-15T02:33:11.086ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
33592025-01-15T02:33:11.178ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a49be6ff-1421-48b9-aa1c-1bab47942cd9", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33602025-01-15T02:33:11.178ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
33612025-01-15T02:33:11.178ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33622025-01-15T02:33:11.179ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "04742362-0eb1-43c5-bca2-4884ac7f91d3", "content-length": "682", "date": "Wed, 15 Jan 2025 02:33:10 GMT"} })
33632025-01-15T02:33:11.179ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33642025-01-15T02:33:11.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33652025-01-15T02:33:11.182ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33662025-01-15T02:33:11.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33672025-01-15T02:33:11.186ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33682025-01-15T02:33:11.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33692025-01-15T02:33:11.188ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33702025-01-15T02:33:11.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33712025-01-15T02:33:11.195ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 689.571477ms iteration = 13 status = {}
33722025-01-15T02:33:11.486ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f1de745c-9d5a-4c09-83b3-c8cb645a5b17 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33732025-01-15T02:33:11.486ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f1de745c-9d5a-4c09-83b3-c8cb645a5b17 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33742025-01-15T02:33:11.488ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f1de745c-9d5a-4c09-83b3-c8cb645a5b17 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33752025-01-15T02:33:12.509ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d60c6fab-695a-4132-a980-5e613c064aff roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33762025-01-15T02:33:12.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d60c6fab-695a-4132-a980-5e613c064aff resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33772025-01-15T02:33:12.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2800 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d60c6fab-695a-4132-a980-5e613c064aff response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33782025-01-15T02:33:13.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b16fc17d-807a-4523-954e-62bf380bf201 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33792025-01-15T02:33:13.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b16fc17d-807a-4523-954e-62bf380bf201 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33802025-01-15T02:33:13.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b16fc17d-807a-4523-954e-62bf380bf201 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33812025-01-15T02:33:14.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ad6de9b-93ed-48b1-ac74-c497d795be9e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33822025-01-15T02:33:14.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ad6de9b-93ed-48b1-ac74-c497d795be9e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33832025-01-15T02:33:14.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2213 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ad6de9b-93ed-48b1-ac74-c497d795be9e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33842025-01-15T02:33:15.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e77d8d8c-60fa-47dc-aa63-bff559e66efd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33852025-01-15T02:33:15.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e77d8d8c-60fa-47dc-aa63-bff559e66efd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33862025-01-15T02:33:15.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2038 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e77d8d8c-60fa-47dc-aa63-bff559e66efd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33872025-01-15T02:33:15.737ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:48700
33882025-01-15T02:33:15.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 48e33e90-33c8-4a0d-ab01-1f28b93a5603 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
33892025-01-15T02:33:15.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 48e33e90-33c8-4a0d-ab01-1f28b93a5603 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
33902025-01-15T02:33:15.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3055 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 48e33e90-33c8-4a0d-ab01-1f28b93a5603 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
33912025-01-15T02:33:15.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e078d701-5f59-4d88-8e1b-927b2aad6d4f roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
33922025-01-15T02:33:15.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e078d701-5f59-4d88-8e1b-927b2aad6d4f resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
33932025-01-15T02:33:15.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2089 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e078d701-5f59-4d88-8e1b-927b2aad6d4f response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
33942025-01-15T02:33:15.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 496 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = ece1e403-40f3-4a5a-8cdf-2b6b7331b2bf response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
33952025-01-15T02:33:16.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 180eb569-7515-41b6-8c98-34a8416d8ad7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33962025-01-15T02:33:16.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 180eb569-7515-41b6-8c98-34a8416d8ad7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33972025-01-15T02:33:16.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1933 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 180eb569-7515-41b6-8c98-34a8416d8ad7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33982025-01-15T02:33:17.619ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 60ca425d-96ea-415d-a3ce-ecc0fdf2d283 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
33992025-01-15T02:33:17.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 60ca425d-96ea-415d-a3ce-ecc0fdf2d283 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34002025-01-15T02:33:17.621ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2024 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 60ca425d-96ea-415d-a3ce-ecc0fdf2d283 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34012025-01-15T02:33:18.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f9c678c8-c797-4e2a-aec2-bd7e7af11ca0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34022025-01-15T02:33:18.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f9c678c8-c797-4e2a-aec2-bd7e7af11ca0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34032025-01-15T02:33:18.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2567 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f9c678c8-c797-4e2a-aec2-bd7e7af11ca0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34042025-01-15T02:33:19.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be32dd8b-c80f-4150-9a0d-64a9e056dc2d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34052025-01-15T02:33:19.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be32dd8b-c80f-4150-9a0d-64a9e056dc2d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34062025-01-15T02:33:19.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1919 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be32dd8b-c80f-4150-9a0d-64a9e056dc2d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34072025-01-15T02:33:20.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
34082025-01-15T02:33:20.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
34092025-01-15T02:33:20.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34102025-01-15T02:33:20.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34112025-01-15T02:33:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34122025-01-15T02:33:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34132025-01-15T02:33:20.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34142025-01-15T02:33:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34152025-01-15T02:33:20.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34162025-01-15T02:33:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34172025-01-15T02:33:20.540ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34182025-01-15T02:33:20.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34192025-01-15T02:33:20.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34202025-01-15T02:33:20.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34212025-01-15T02:33:20.570ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.437659ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34222025-01-15T02:33:20.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34232025-01-15T02:33:20.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
34242025-01-15T02:33:20.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34252025-01-15T02:33:20.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34262025-01-15T02:33:20.590ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
34272025-01-15T02:33:20.590ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.696606ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
34282025-01-15T02:33:20.686ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 15d030b9-8be4-4b5c-9c1f-0b4d36165004 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34292025-01-15T02:33:20.686ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 15d030b9-8be4-4b5c-9c1f-0b4d36165004 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34302025-01-15T02:33:20.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1457 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 15d030b9-8be4-4b5c-9c1f-0b4d36165004 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34312025-01-15T02:33:21.708ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6464fedd-f6af-4c93-b632-8c7ec0a7ab5f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34322025-01-15T02:33:21.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6464fedd-f6af-4c93-b632-8c7ec0a7ab5f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34332025-01-15T02:33:21.711ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6464fedd-f6af-4c93-b632-8c7ec0a7ab5f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34342025-01-15T02:33:22.731ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 05b40859-6335-4d9c-9d40-f329b27bcfb9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34352025-01-15T02:33:22.731ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 05b40859-6335-4d9c-9d40-f329b27bcfb9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34362025-01-15T02:33:22.733ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2151 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 05b40859-6335-4d9c-9d40-f329b27bcfb9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34372025-01-15T02:33:23.754ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 34530627-a47d-4682-b005-9a950ee68a44 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34382025-01-15T02:33:23.754ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 34530627-a47d-4682-b005-9a950ee68a44 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34392025-01-15T02:33:23.756ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2006 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 34530627-a47d-4682-b005-9a950ee68a44 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34402025-01-15T02:33:24.777ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9781b827-5b65-452c-90dc-ba4984b6d1d3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34412025-01-15T02:33:24.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9781b827-5b65-452c-90dc-ba4984b6d1d3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34422025-01-15T02:33:24.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2869 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9781b827-5b65-452c-90dc-ba4984b6d1d3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34432025-01-15T02:33:25.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 423 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 8450b0e1-bacb-4887-a73a-1e59686c8722 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
34442025-01-15T02:33:25.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1469b390-1426-4a53-9d20-00ef2787d29b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34452025-01-15T02:33:25.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1469b390-1426-4a53-9d20-00ef2787d29b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34462025-01-15T02:33:25.801ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1468 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1469b390-1426-4a53-9d20-00ef2787d29b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34472025-01-15T02:33:26.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 81cc0f12-78d5-4222-8b34-d0967d473ac8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34482025-01-15T02:33:26.820ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 81cc0f12-78d5-4222-8b34-d0967d473ac8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34492025-01-15T02:33:26.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2358 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 81cc0f12-78d5-4222-8b34-d0967d473ac8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34502025-01-15T02:33:27.843ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cdccdc89-7cea-4377-aa32-53cab9b82200 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34512025-01-15T02:33:27.843ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cdccdc89-7cea-4377-aa32-53cab9b82200 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34522025-01-15T02:33:27.845ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2404 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cdccdc89-7cea-4377-aa32-53cab9b82200 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34532025-01-15T02:33:28.866ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2bba73ef-6d65-4aba-99d6-13a985f6a4e6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34542025-01-15T02:33:28.866ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2bba73ef-6d65-4aba-99d6-13a985f6a4e6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34552025-01-15T02:33:28.868ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2223 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2bba73ef-6d65-4aba-99d6-13a985f6a4e6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34562025-01-15T02:33:29.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 036a8395-19a7-49df-bd5b-180066d4fb5c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34572025-01-15T02:33:29.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 036a8395-19a7-49df-bd5b-180066d4fb5c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34582025-01-15T02:33:29.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 036a8395-19a7-49df-bd5b-180066d4fb5c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34592025-01-15T02:33:30.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
34602025-01-15T02:33:30.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
34612025-01-15T02:33:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34622025-01-15T02:33:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34632025-01-15T02:33:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34642025-01-15T02:33:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34652025-01-15T02:33:30.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34662025-01-15T02:33:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34672025-01-15T02:33:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34682025-01-15T02:33:30.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.380631ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34692025-01-15T02:33:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34702025-01-15T02:33:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34712025-01-15T02:33:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34722025-01-15T02:33:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34732025-01-15T02:33:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34742025-01-15T02:33:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34752025-01-15T02:33:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
34762025-01-15T02:33:30.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34772025-01-15T02:33:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34782025-01-15T02:33:30.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
34792025-01-15T02:33:30.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.741027ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
34802025-01-15T02:33:30.908ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0154ee13-47af-4e3e-bd2c-2a187cafabc8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34812025-01-15T02:33:30.908ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0154ee13-47af-4e3e-bd2c-2a187cafabc8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34822025-01-15T02:33:30.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0154ee13-47af-4e3e-bd2c-2a187cafabc8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34832025-01-15T02:33:31.929ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b0386ac3-1699-4fe4-b921-08066749ae07 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34842025-01-15T02:33:31.929ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b0386ac3-1699-4fe4-b921-08066749ae07 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34852025-01-15T02:33:31.931ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b0386ac3-1699-4fe4-b921-08066749ae07 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34862025-01-15T02:33:32.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b2db917-0318-4847-abb0-8fcc2ae7ba9e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34872025-01-15T02:33:32.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b2db917-0318-4847-abb0-8fcc2ae7ba9e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34882025-01-15T02:33:32.952ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2724 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b2db917-0318-4847-abb0-8fcc2ae7ba9e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34892025-01-15T02:33:33.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 03cfe33e-da39-45d5-ad52-6319e0bc5ed4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34902025-01-15T02:33:33.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 03cfe33e-da39-45d5-ad52-6319e0bc5ed4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34912025-01-15T02:33:33.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2434 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 03cfe33e-da39-45d5-ad52-6319e0bc5ed4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34922025-01-15T02:33:34.995ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7569a69e-3c37-492a-a8ae-43f1cd8dda04 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34932025-01-15T02:33:34.995ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7569a69e-3c37-492a-a8ae-43f1cd8dda04 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34942025-01-15T02:33:34.997ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1805 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7569a69e-3c37-492a-a8ae-43f1cd8dda04 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34952025-01-15T02:33:35.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 450 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = c7b22b5a-6d7d-4043-b0f3-cad87cd64ba4 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
34962025-01-15T02:33:36.019ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ef842e56-d05e-4443-bb6f-ae8083a3931c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34972025-01-15T02:33:36.019ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ef842e56-d05e-4443-bb6f-ae8083a3931c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34982025-01-15T02:33:36.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2328 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ef842e56-d05e-4443-bb6f-ae8083a3931c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
34992025-01-15T02:33:37.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4882e96f-f5a1-4621-854b-0c799011e082 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35002025-01-15T02:33:37.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4882e96f-f5a1-4621-854b-0c799011e082 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35012025-01-15T02:33:37.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2067 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4882e96f-f5a1-4621-854b-0c799011e082 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35022025-01-15T02:33:38.062ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 77df445e-e8e8-47fe-b52a-98d340f11605 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35032025-01-15T02:33:38.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 77df445e-e8e8-47fe-b52a-98d340f11605 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35042025-01-15T02:33:38.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2187 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 77df445e-e8e8-47fe-b52a-98d340f11605 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35052025-01-15T02:33:39.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6909efb3-05b7-4bdf-ae34-c9d63128984b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35062025-01-15T02:33:39.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6909efb3-05b7-4bdf-ae34-c9d63128984b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35072025-01-15T02:33:39.087ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2168 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6909efb3-05b7-4bdf-ae34-c9d63128984b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35082025-01-15T02:33:40.129ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e9161c8a-41c2-4255-8737-6781755e1c24 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35092025-01-15T02:33:40.129ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e9161c8a-41c2-4255-8737-6781755e1c24 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35102025-01-15T02:33:40.131ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1979 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e9161c8a-41c2-4255-8737-6781755e1c24 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
35112025-01-15T02:33:40.504ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
35122025-01-15T02:33:40.504ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35132025-01-15T02:33:40.504ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
35142025-01-15T02:33:40.504ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
35152025-01-15T02:33:40.504ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
35162025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35172025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
35182025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
35192025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
35202025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
35212025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Timeout
35222025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35232025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35242025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35252025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35262025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
35272025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
35282025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35292025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
35302025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35312025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
35322025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35332025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
35342025-01-15T02:33:40.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
35352025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
35362025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35372025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35382025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
35392025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35402025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
35412025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35422025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35432025-01-15T02:33:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35442025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35452025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35462025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35472025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35482025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35492025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35502025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35512025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35522025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35532025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
35542025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
35552025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
35562025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
35572025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
35582025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35592025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
35602025-01-15T02:33:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35612025-01-15T02:33:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
35622025-01-15T02:33:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
35632025-01-15T02:33:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35642025-01-15T02:33:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35652025-01-15T02:33:40.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3707, tv_nsec: 343386631 } })
35662025-01-15T02:33:40.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
35672025-01-15T02:33:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35682025-01-15T02:33:40.508ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.832164ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
35692025-01-15T02:33:40.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35702025-01-15T02:33:40.508ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3707, tv_nsec: 344785208 } })
35712025-01-15T02:33:40.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
35722025-01-15T02:33:40.509ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35732025-01-15T02:33:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35742025-01-15T02:33:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35752025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35762025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35772025-01-15T02:33:40.511ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35782025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
35792025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35802025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35812025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35822025-01-15T02:33:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35832025-01-15T02:33:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
35842025-01-15T02:33:40.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35852025-01-15T02:33:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35862025-01-15T02:33:40.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35872025-01-15T02:33:40.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35882025-01-15T02:33:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
35892025-01-15T02:33:40.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35902025-01-15T02:33:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35912025-01-15T02:33:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
35922025-01-15T02:33:40.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35932025-01-15T02:33:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35942025-01-15T02:33:40.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35952025-01-15T02:33:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35962025-01-15T02:33:40.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35972025-01-15T02:33:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35982025-01-15T02:33:40.520ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35992025-01-15T02:33:40.520ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
36002025-01-15T02:33:40.520ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36012025-01-15T02:33:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36022025-01-15T02:33:40.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3707, tv_nsec: 357648595 } })
36032025-01-15T02:33:40.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
36042025-01-15T02:33:40.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36052025-01-15T02:33:40.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36062025-01-15T02:33:40.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36072025-01-15T02:33:40.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36082025-01-15T02:33:40.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36092025-01-15T02:33:40.550ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36102025-01-15T02:33:40.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36112025-01-15T02:33:40.551ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36122025-01-15T02:33:40.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36132025-01-15T02:33:40.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36142025-01-15T02:33:40.551ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 44.884511ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
36152025-01-15T02:33:40.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36162025-01-15T02:33:40.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36172025-01-15T02:33:40.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36182025-01-15T02:33:40.554ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36192025-01-15T02:33:40.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36202025-01-15T02:33:40.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36212025-01-15T02:33:40.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36222025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36232025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
36242025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36252025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36262025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36272025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36282025-01-15T02:33:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36292025-01-15T02:33:40.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.106969ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
36302025-01-15T02:33:40.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 52.005123ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
36312025-01-15T02:33:40.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36322025-01-15T02:33:40.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36332025-01-15T02:33:40.561ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36342025-01-15T02:33:40.561ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
36352025-01-15T02:33:40.561ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 56.146365ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
36362025-01-15T02:33:40.565ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36372025-01-15T02:33:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36382025-01-15T02:33:40.567ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36392025-01-15T02:33:40.567ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36402025-01-15T02:33:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
36412025-01-15T02:33:40.569ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36422025-01-15T02:33:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
36432025-01-15T02:33:40.569ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36442025-01-15T02:33:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36452025-01-15T02:33:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36462025-01-15T02:33:40.570ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
36472025-01-15T02:33:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.751127ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
36482025-01-15T02:33:40.578ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
36492025-01-15T02:33:40.578ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 72.662831ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
36502025-01-15T02:33:40.593ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36512025-01-15T02:33:40.593ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36522025-01-15T02:33:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36532025-01-15T02:33:40.595ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36542025-01-15T02:33:40.595ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36552025-01-15T02:33:40.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36562025-01-15T02:33:40.596ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 90.778539ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
36572025-01-15T02:33:40.603ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 97.45644ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36582025-01-15T02:33:40.605ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36592025-01-15T02:33:40.608ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
36602025-01-15T02:33:40.608ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36612025-01-15T02:33:40.608ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
36622025-01-15T02:33:40.608ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 102.757793ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
36632025-01-15T02:33:40.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36642025-01-15T02:33:40.610ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 105.065655ms iteration = 14 status = 0
36652025-01-15T02:33:40.620ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
36662025-01-15T02:33:40.620ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36672025-01-15T02:33:40.620ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4128e5f4-2139-4cda-b018-44e87d3fc287", "content-length": "26", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
36682025-01-15T02:33:40.620ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36692025-01-15T02:33:40.623ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
36702025-01-15T02:33:40.633ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 126.683758ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
36712025-01-15T02:33:40.647ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36722025-01-15T02:33:40.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36732025-01-15T02:33:40.672ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.237203ms iteration = 16 status = {"physical_disks_added":0}
36742025-01-15T02:33:40.701ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
36752025-01-15T02:33:40.701ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36762025-01-15T02:33:40.701ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddd71a3a-eb7d-47e7-8925-706d7db633e7", "content-length": "26", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
36772025-01-15T02:33:40.701ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36782025-01-15T02:33:40.703ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36792025-01-15T02:33:40.725ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
36802025-01-15T02:33:40.725ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36812025-01-15T02:33:40.725ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "649ae49a-7066-4664-ab48-c41ba292c150", "content-length": "26", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
36822025-01-15T02:33:40.725ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36832025-01-15T02:33:40.727ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36842025-01-15T02:33:40.737ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36852025-01-15T02:33:40.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36862025-01-15T02:33:40.739ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36872025-01-15T02:33:40.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36882025-01-15T02:33:40.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36892025-01-15T02:33:40.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36902025-01-15T02:33:40.845ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 340.788396ms iteration = 14 status = {}
36912025-01-15T02:33:40.867ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
36922025-01-15T02:33:40.867ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36932025-01-15T02:33:40.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36942025-01-15T02:33:40.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36952025-01-15T02:33:40.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36962025-01-15T02:33:40.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36972025-01-15T02:33:40.875ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
36982025-01-15T02:33:40.875ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecafbe77-9424-4762-82c3-c72efcb9f34a", "content-length": "1", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
36992025-01-15T02:33:40.876ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37002025-01-15T02:33:40.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
37012025-01-15T02:33:40.879ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 375.103722ms iteration = 14 status = null
37022025-01-15T02:33:40.893ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37032025-01-15T02:33:40.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37042025-01-15T02:33:40.895ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
37052025-01-15T02:33:40.896ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37062025-01-15T02:33:40.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37072025-01-15T02:33:40.905ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
37082025-01-15T02:33:40.905ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "7832ab5c-1b13-40a4-9484-7db6be439bd4", "content-length": "134", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37092025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
37102025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
37112025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
37122025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
37132025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37142025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37152025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37162025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37172025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37182025-01-15T02:33:40.905ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37192025-01-15T02:33:40.906ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
37202025-01-15T02:33:40.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37212025-01-15T02:33:40.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37222025-01-15T02:33:40.908ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
37232025-01-15T02:33:40.908ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0159ec2-2f3e-4ce6-81a9-ec91e612b7f6", "content-length": "2", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37242025-01-15T02:33:40.908ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
37252025-01-15T02:33:40.908ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d1ea21c-ad8d-4fd7-9ce0-f79d1b6e93b0", "content-length": "52", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37262025-01-15T02:33:40.908ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
37272025-01-15T02:33:40.908ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
37282025-01-15T02:33:40.956ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
37292025-01-15T02:33:40.961ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "98355f6a-e71e-4227-9188-011573f065d9", "content-length": "372", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37302025-01-15T02:33:40.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37312025-01-15T02:33:40.961ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5f832cd-55c4-4c4e-9c70-a536dfb37ae1", "content-length": "691", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37322025-01-15T02:33:40.961ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
37332025-01-15T02:33:40.961ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
37342025-01-15T02:33:40.961ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
37352025-01-15T02:33:40.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37362025-01-15T02:33:40.961ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 456.010336ms iteration = 14 status = {}
37372025-01-15T02:33:41.002ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
37382025-01-15T02:33:41.002ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
37392025-01-15T02:33:41.003ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7ac67c6c-66be-478f-8321-25afdef67b33", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37402025-01-15T02:33:41.003ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37412025-01-15T02:33:41.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37422025-01-15T02:33:41.005ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37432025-01-15T02:33:41.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37442025-01-15T02:33:41.009ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37452025-01-15T02:33:41.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37462025-01-15T02:33:41.012ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
37472025-01-15T02:33:41.012ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37482025-01-15T02:33:41.015ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fe6cf71a-c84c-4a11-abc6-775ad26f62ba", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37492025-01-15T02:33:41.017ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 511.504992ms iteration = 14 status = {}
37502025-01-15T02:33:41.055ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "17d4a8bf-54ee-41d2-a76b-f1669908ba12", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37512025-01-15T02:33:41.055ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
37522025-01-15T02:33:41.055ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
37532025-01-15T02:33:41.055ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae627ff5-996e-4df2-9906-6763631f4168", "content-length": "682", "date": "Wed, 15 Jan 2025 02:33:40 GMT"} })
37542025-01-15T02:33:41.055ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37552025-01-15T02:33:41.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37562025-01-15T02:33:41.059ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37572025-01-15T02:33:41.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37582025-01-15T02:33:41.061ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37592025-01-15T02:33:41.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37602025-01-15T02:33:41.063ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37612025-01-15T02:33:41.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37622025-01-15T02:33:41.071ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 566.100836ms iteration = 14 status = {}
37632025-01-15T02:33:41.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c860a0ed-7916-48a7-947e-528366325b72 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37642025-01-15T02:33:41.150ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c860a0ed-7916-48a7-947e-528366325b72 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37652025-01-15T02:33:41.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2286 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c860a0ed-7916-48a7-947e-528366325b72 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37662025-01-15T02:33:42.171ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a15a8e38-2e88-4c1f-9dc4-acb46edefdbd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37672025-01-15T02:33:42.171ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a15a8e38-2e88-4c1f-9dc4-acb46edefdbd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37682025-01-15T02:33:42.173ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2099 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a15a8e38-2e88-4c1f-9dc4-acb46edefdbd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37692025-01-15T02:33:43.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 21e61eb5-35db-46e6-98a8-474b736ba7fd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37702025-01-15T02:33:43.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 21e61eb5-35db-46e6-98a8-474b736ba7fd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37712025-01-15T02:33:43.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 21e61eb5-35db-46e6-98a8-474b736ba7fd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37722025-01-15T02:33:44.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4542f6ab-3030-4391-8adf-d884d035af1f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37732025-01-15T02:33:44.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4542f6ab-3030-4391-8adf-d884d035af1f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37742025-01-15T02:33:44.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2142 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4542f6ab-3030-4391-8adf-d884d035af1f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37752025-01-15T02:33:45.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e52757fd-d3e2-4e99-a337-9b7bb4589e09 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37762025-01-15T02:33:45.235ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e52757fd-d3e2-4e99-a337-9b7bb4589e09 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37772025-01-15T02:33:45.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2083 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e52757fd-d3e2-4e99-a337-9b7bb4589e09 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37782025-01-15T02:33:45.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 422 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = cb28e2f3-3ff5-457c-83ed-d69f5de614ae response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
37792025-01-15T02:33:46.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfb5c407-92d0-47e9-90bd-dbdaacf028b1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37802025-01-15T02:33:46.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfb5c407-92d0-47e9-90bd-dbdaacf028b1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37812025-01-15T02:33:46.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfb5c407-92d0-47e9-90bd-dbdaacf028b1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37822025-01-15T02:33:47.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 063ea656-9d71-4e62-b5bf-219e6e0f14c5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37832025-01-15T02:33:47.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 063ea656-9d71-4e62-b5bf-219e6e0f14c5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37842025-01-15T02:33:47.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2230 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 063ea656-9d71-4e62-b5bf-219e6e0f14c5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37852025-01-15T02:33:48.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a610a564-31c9-4606-9ec4-a2d699423e39 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37862025-01-15T02:33:48.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a610a564-31c9-4606-9ec4-a2d699423e39 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37872025-01-15T02:33:48.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1881 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a610a564-31c9-4606-9ec4-a2d699423e39 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37882025-01-15T02:33:49.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc73bd5d-a131-4097-a5db-6c6e8f5ada57 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37892025-01-15T02:33:49.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc73bd5d-a131-4097-a5db-6c6e8f5ada57 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37902025-01-15T02:33:49.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2403 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc73bd5d-a131-4097-a5db-6c6e8f5ada57 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37912025-01-15T02:33:50.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2143751e-eaaa-4545-b190-a1806392a536 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37922025-01-15T02:33:50.341ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2143751e-eaaa-4545-b190-a1806392a536 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37932025-01-15T02:33:50.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2144 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2143751e-eaaa-4545-b190-a1806392a536 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
37942025-01-15T02:33:50.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
37952025-01-15T02:33:50.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
37962025-01-15T02:33:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37972025-01-15T02:33:50.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37982025-01-15T02:33:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37992025-01-15T02:33:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38002025-01-15T02:33:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38012025-01-15T02:33:50.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38022025-01-15T02:33:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38032025-01-15T02:33:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38042025-01-15T02:33:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38052025-01-15T02:33:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38062025-01-15T02:33:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38072025-01-15T02:33:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38082025-01-15T02:33:50.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38092025-01-15T02:33:50.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
38102025-01-15T02:33:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38112025-01-15T02:33:50.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38122025-01-15T02:33:50.528ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
38132025-01-15T02:33:50.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.163829ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
38142025-01-15T02:33:50.535ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.517697ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38152025-01-15T02:33:51.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 290c857f-7b53-4415-8b78-91954a974003 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38162025-01-15T02:33:51.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 290c857f-7b53-4415-8b78-91954a974003 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38172025-01-15T02:33:51.365ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1906 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 290c857f-7b53-4415-8b78-91954a974003 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38182025-01-15T02:33:52.385ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2feebcb5-eb2d-4fd9-a6bf-98fdc31f5be9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38192025-01-15T02:33:52.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2feebcb5-eb2d-4fd9-a6bf-98fdc31f5be9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38202025-01-15T02:33:52.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2065 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2feebcb5-eb2d-4fd9-a6bf-98fdc31f5be9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38212025-01-15T02:33:53.407ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1707c63c-6cd6-4a13-9cb8-e8f0ba19e17b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38222025-01-15T02:33:53.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1707c63c-6cd6-4a13-9cb8-e8f0ba19e17b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38232025-01-15T02:33:53.409ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2342 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1707c63c-6cd6-4a13-9cb8-e8f0ba19e17b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38242025-01-15T02:33:54.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7e663fd9-2e3c-4267-9a4e-ed554d3cd2bc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38252025-01-15T02:33:54.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7e663fd9-2e3c-4267-9a4e-ed554d3cd2bc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38262025-01-15T02:33:54.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7e663fd9-2e3c-4267-9a4e-ed554d3cd2bc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38272025-01-15T02:33:55.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7d145d47-f292-474e-8a5f-608f4b95454b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38282025-01-15T02:33:55.450ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7d145d47-f292-474e-8a5f-608f4b95454b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38292025-01-15T02:33:55.452ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2089 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7d145d47-f292-474e-8a5f-608f4b95454b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38302025-01-15T02:33:55.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 496 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = e614f9e7-46df-42da-9db6-159624933b8b response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
38312025-01-15T02:33:56.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3674c71a-cc0b-491f-b72c-b463ecfee7b0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38322025-01-15T02:33:56.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3674c71a-cc0b-491f-b72c-b463ecfee7b0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38332025-01-15T02:33:56.474ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1881 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3674c71a-cc0b-491f-b72c-b463ecfee7b0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38342025-01-15T02:33:57.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bee6cbc5-a741-4d70-9feb-c0cffe3065c2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38352025-01-15T02:33:57.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bee6cbc5-a741-4d70-9feb-c0cffe3065c2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38362025-01-15T02:33:57.495ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1918 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bee6cbc5-a741-4d70-9feb-c0cffe3065c2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38372025-01-15T02:33:58.516ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c9346ea4-64ec-4ec9-b13c-1fd446697a27 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38382025-01-15T02:33:58.516ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c9346ea4-64ec-4ec9-b13c-1fd446697a27 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38392025-01-15T02:33:58.518ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2327 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c9346ea4-64ec-4ec9-b13c-1fd446697a27 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38402025-01-15T02:33:59.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0386b736-af16-45e0-b776-9f5a09401581 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38412025-01-15T02:33:59.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0386b736-af16-45e0-b776-9f5a09401581 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38422025-01-15T02:33:59.539ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1444 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0386b736-af16-45e0-b776-9f5a09401581 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38432025-01-15T02:34:00.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
38442025-01-15T02:34:00.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
38452025-01-15T02:34:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38462025-01-15T02:34:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38472025-01-15T02:34:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38482025-01-15T02:34:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38492025-01-15T02:34:00.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38502025-01-15T02:34:00.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38512025-01-15T02:34:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38522025-01-15T02:34:00.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.203123ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38532025-01-15T02:34:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38542025-01-15T02:34:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38552025-01-15T02:34:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38562025-01-15T02:34:00.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38572025-01-15T02:34:00.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38582025-01-15T02:34:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38592025-01-15T02:34:00.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
38602025-01-15T02:34:00.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38612025-01-15T02:34:00.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38622025-01-15T02:34:00.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
38632025-01-15T02:34:00.524ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.654364ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
38642025-01-15T02:34:00.559ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 38ea87df-6fd7-47d0-89a5-781cfea870f4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38652025-01-15T02:34:00.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 38ea87df-6fd7-47d0-89a5-781cfea870f4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38662025-01-15T02:34:00.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1674 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 38ea87df-6fd7-47d0-89a5-781cfea870f4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38672025-01-15T02:34:01.582ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3835d902-a052-4943-91c4-eee539785bb8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38682025-01-15T02:34:01.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3835d902-a052-4943-91c4-eee539785bb8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38692025-01-15T02:34:01.584ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2061 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3835d902-a052-4943-91c4-eee539785bb8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38702025-01-15T02:34:02.605ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 255e53f0-1e5b-4d98-a08a-9fc948436228 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38712025-01-15T02:34:02.606ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 255e53f0-1e5b-4d98-a08a-9fc948436228 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38722025-01-15T02:34:02.607ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2076 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 255e53f0-1e5b-4d98-a08a-9fc948436228 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38732025-01-15T02:34:03.628ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5924e62b-c0c0-4406-b157-0fa2110e943b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38742025-01-15T02:34:03.628ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5924e62b-c0c0-4406-b157-0fa2110e943b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38752025-01-15T02:34:03.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1508 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5924e62b-c0c0-4406-b157-0fa2110e943b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38762025-01-15T02:34:04.649ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b87abd37-af02-47eb-97a6-8d7a7d8d1bcc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38772025-01-15T02:34:04.649ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b87abd37-af02-47eb-97a6-8d7a7d8d1bcc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38782025-01-15T02:34:04.651ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1901 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b87abd37-af02-47eb-97a6-8d7a7d8d1bcc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38792025-01-15T02:34:05.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a66ddf70-b268-41dc-b6e2-5978329ea9d2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38802025-01-15T02:34:05.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a66ddf70-b268-41dc-b6e2-5978329ea9d2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38812025-01-15T02:34:05.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1960 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a66ddf70-b268-41dc-b6e2-5978329ea9d2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38822025-01-15T02:34:05.748ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 472 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 8af05580-2a7b-4608-b6ff-e63b9dafae24 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
38832025-01-15T02:34:06.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f68d9567-515e-47d0-9c19-5176d04a6dcf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38842025-01-15T02:34:06.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f68d9567-515e-47d0-9c19-5176d04a6dcf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38852025-01-15T02:34:06.694ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2581 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f68d9567-515e-47d0-9c19-5176d04a6dcf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38862025-01-15T02:34:07.714ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 53b0d185-c065-47d7-91b4-0a2172bc4217 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38872025-01-15T02:34:07.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 53b0d185-c065-47d7-91b4-0a2172bc4217 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38882025-01-15T02:34:07.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1889 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 53b0d185-c065-47d7-91b4-0a2172bc4217 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38892025-01-15T02:34:08.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8fc162f1-b3ac-4b1a-ba78-b37cd452cf91 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38902025-01-15T02:34:08.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8fc162f1-b3ac-4b1a-ba78-b37cd452cf91 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38912025-01-15T02:34:08.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1885 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8fc162f1-b3ac-4b1a-ba78-b37cd452cf91 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38922025-01-15T02:34:09.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f47348c6-4032-454f-97a3-716b907f30bf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38932025-01-15T02:34:09.759ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f47348c6-4032-454f-97a3-716b907f30bf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38942025-01-15T02:34:09.761ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f47348c6-4032-454f-97a3-716b907f30bf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
38952025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
38962025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
38972025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
38982025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
38992025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39002025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
39012025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
39022025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
39032025-01-15T02:34:10.505ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:24:10.505199386 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
39042025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
39052025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
39062025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
39072025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
39082025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
39092025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
39102025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39112025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
39122025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39132025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39142025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
39152025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39162025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39172025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
39182025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39192025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
39202025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
39212025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
39222025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39232025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39242025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39252025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39262025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
39272025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39282025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
39292025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39302025-01-15T02:34:10.505ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
39312025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
39322025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
39332025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
39342025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
39352025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
39362025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39372025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39382025-01-15T02:34:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
39392025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
39402025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
39412025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
39422025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39432025-01-15T02:34:10.506ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
39442025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
39452025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39462025-01-15T02:34:10.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
39472025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.451\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
39482025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39492025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
39502025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
39512025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39522025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
39532025-01-15T02:34:10.506ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3737, tv_nsec: 342043594 } })
39542025-01-15T02:34:10.506ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3737, tv_nsec: 342052255 } })
39552025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
39562025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39572025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39582025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39592025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39602025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39612025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
39622025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39632025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39642025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39652025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39662025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39672025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
39682025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
39692025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
39702025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39712025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.261715ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
39722025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
39732025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
39742025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
39752025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
39762025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
39772025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39782025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39792025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
39802025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
39812025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
39822025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
39832025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
39842025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.425498ms iteration = 8 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
39852025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
39862025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
39872025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39882025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39892025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
39902025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39912025-01-15T02:34:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
39922025-01-15T02:34:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39932025-01-15T02:34:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39942025-01-15T02:34:10.508ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39952025-01-15T02:34:10.509ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.027619ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
39962025-01-15T02:34:10.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3737, tv_nsec: 345784126 } })
39972025-01-15T02:34:10.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
39982025-01-15T02:34:10.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
39992025-01-15T02:34:10.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40002025-01-15T02:34:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40012025-01-15T02:34:10.511ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3737, tv_nsec: 347350748 } })
40022025-01-15T02:34:10.511ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
40032025-01-15T02:34:10.515ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40042025-01-15T02:34:10.517ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:24:10.505199386 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
40052025-01-15T02:34:10.517ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.538869ms iteration = 8 status = {"expiration":"2025-01-15T02:24:10.505199386Z","pruned":[]}
40062025-01-15T02:34:10.518ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40072025-01-15T02:34:10.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40082025-01-15T02:34:10.518ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40092025-01-15T02:34:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40102025-01-15T02:34:10.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40112025-01-15T02:34:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40122025-01-15T02:34:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
40132025-01-15T02:34:10.519ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
40142025-01-15T02:34:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
40152025-01-15T02:34:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
40162025-01-15T02:34:10.520ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
40172025-01-15T02:34:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
40182025-01-15T02:34:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
40192025-01-15T02:34:10.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40202025-01-15T02:34:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40212025-01-15T02:34:10.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40222025-01-15T02:34:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40232025-01-15T02:34:10.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40242025-01-15T02:34:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40252025-01-15T02:34:10.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40262025-01-15T02:34:10.526ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.120723ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
40272025-01-15T02:34:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40282025-01-15T02:34:10.531ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40292025-01-15T02:34:10.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40302025-01-15T02:34:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40312025-01-15T02:34:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40322025-01-15T02:34:10.534ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40332025-01-15T02:34:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40342025-01-15T02:34:10.538ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40352025-01-15T02:34:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 32.693622ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40362025-01-15T02:34:10.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40372025-01-15T02:34:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40382025-01-15T02:34:10.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40392025-01-15T02:34:10.567ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40402025-01-15T02:34:10.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40412025-01-15T02:34:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40422025-01-15T02:34:10.569ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40432025-01-15T02:34:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
40442025-01-15T02:34:10.572ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40452025-01-15T02:34:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40462025-01-15T02:34:10.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.360711ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
40472025-01-15T02:34:10.572ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40482025-01-15T02:34:10.572ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
40492025-01-15T02:34:10.572ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40502025-01-15T02:34:10.573ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40512025-01-15T02:34:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40522025-01-15T02:34:10.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40532025-01-15T02:34:10.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40542025-01-15T02:34:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40552025-01-15T02:34:10.575ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40562025-01-15T02:34:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40572025-01-15T02:34:10.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40582025-01-15T02:34:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40592025-01-15T02:34:10.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40602025-01-15T02:34:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40612025-01-15T02:34:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
40622025-01-15T02:34:10.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40632025-01-15T02:34:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40642025-01-15T02:34:10.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40652025-01-15T02:34:10.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40662025-01-15T02:34:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40672025-01-15T02:34:10.578ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40682025-01-15T02:34:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40692025-01-15T02:34:10.579ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40702025-01-15T02:34:10.579ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.098572ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
40712025-01-15T02:34:10.579ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
40722025-01-15T02:34:10.580ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
40732025-01-15T02:34:10.580ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 73.563286ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
40742025-01-15T02:34:10.580ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3737, tv_nsec: 416917925 } })
40752025-01-15T02:34:10.581ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
40762025-01-15T02:34:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
40772025-01-15T02:34:10.581ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40782025-01-15T02:34:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40792025-01-15T02:34:10.582ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
40802025-01-15T02:34:10.582ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
40812025-01-15T02:34:10.582ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
40822025-01-15T02:34:10.582ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40832025-01-15T02:34:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40842025-01-15T02:34:10.584ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40852025-01-15T02:34:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40862025-01-15T02:34:10.585ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40872025-01-15T02:34:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40882025-01-15T02:34:10.586ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40892025-01-15T02:34:10.587ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40902025-01-15T02:34:10.588ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
40912025-01-15T02:34:10.588ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
40922025-01-15T02:34:10.588ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 82.378443ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
40932025-01-15T02:34:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40942025-01-15T02:34:10.590ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40952025-01-15T02:34:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40962025-01-15T02:34:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40972025-01-15T02:34:10.593ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40982025-01-15T02:34:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40992025-01-15T02:34:10.596ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41002025-01-15T02:34:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41012025-01-15T02:34:10.597ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41022025-01-15T02:34:10.598ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
41032025-01-15T02:34:10.598ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.032461ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
41042025-01-15T02:34:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
41052025-01-15T02:34:10.601ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41062025-01-15T02:34:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41072025-01-15T02:34:10.613ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41082025-01-15T02:34:10.614ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
41092025-01-15T02:34:10.614ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41102025-01-15T02:34:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41112025-01-15T02:34:10.615ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41122025-01-15T02:34:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41132025-01-15T02:34:10.620ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
41142025-01-15T02:34:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
41152025-01-15T02:34:10.622ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41162025-01-15T02:34:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 118.463857ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41172025-01-15T02:34:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
41182025-01-15T02:34:10.624ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41192025-01-15T02:34:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41202025-01-15T02:34:10.626ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 120.627795ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41212025-01-15T02:34:10.632ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 126.548285ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
41222025-01-15T02:34:10.635ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
41232025-01-15T02:34:10.635ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
41242025-01-15T02:34:10.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41252025-01-15T02:34:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41262025-01-15T02:34:10.636ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
41272025-01-15T02:34:10.636ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
41282025-01-15T02:34:10.636ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
41292025-01-15T02:34:10.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41302025-01-15T02:34:10.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41312025-01-15T02:34:10.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41322025-01-15T02:34:10.638ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41332025-01-15T02:34:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41342025-01-15T02:34:10.641ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41352025-01-15T02:34:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41362025-01-15T02:34:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41372025-01-15T02:34:10.642ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41382025-01-15T02:34:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41392025-01-15T02:34:10.643ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 136.713729ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41402025-01-15T02:34:10.644ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 138.493937ms iteration = 17 status = {"physical_disks_added":0}
41412025-01-15T02:34:10.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41422025-01-15T02:34:10.648ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41432025-01-15T02:34:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41442025-01-15T02:34:10.648ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
41452025-01-15T02:34:10.648ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
41462025-01-15T02:34:10.648ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
41472025-01-15T02:34:10.648ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 143.601605ms iteration = 8 status = {"generation":2}
41482025-01-15T02:34:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
41492025-01-15T02:34:10.649ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41502025-01-15T02:34:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41512025-01-15T02:34:10.650ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41522025-01-15T02:34:10.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41532025-01-15T02:34:10.651ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
41542025-01-15T02:34:10.652ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
41552025-01-15T02:34:10.652ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 147.186142ms iteration = 15 status = 0
41562025-01-15T02:34:10.653ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41572025-01-15T02:34:10.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41582025-01-15T02:34:10.655ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41592025-01-15T02:34:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41602025-01-15T02:34:10.657ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41612025-01-15T02:34:10.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41622025-01-15T02:34:10.659ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
41632025-01-15T02:34:10.659ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41642025-01-15T02:34:10.659ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49d295bf-d135-4c0d-88ae-8ab65199838c", "content-length": "26", "date": "Wed, 15 Jan 2025 02:34:09 GMT"} })
41652025-01-15T02:34:10.659ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41662025-01-15T02:34:10.661ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41672025-01-15T02:34:10.667ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41682025-01-15T02:34:10.667ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
41692025-01-15T02:34:10.667ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 162.158245ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
41702025-01-15T02:34:10.670ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
41712025-01-15T02:34:10.670ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 164.387316ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
41722025-01-15T02:34:10.685ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
41732025-01-15T02:34:10.685ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
41742025-01-15T02:34:10.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41752025-01-15T02:34:10.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41762025-01-15T02:34:10.686ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41772025-01-15T02:34:10.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41782025-01-15T02:34:10.689ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41792025-01-15T02:34:10.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41802025-01-15T02:34:10.694ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41812025-01-15T02:34:10.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41822025-01-15T02:34:10.696ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41832025-01-15T02:34:10.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41842025-01-15T02:34:10.700ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
41852025-01-15T02:34:10.700ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 195.360511ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
41862025-01-15T02:34:10.701ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
41872025-01-15T02:34:10.701ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 195.66013ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
41882025-01-15T02:34:10.739ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
41892025-01-15T02:34:10.739ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41902025-01-15T02:34:10.740ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "513a431b-1e31-4bf8-bdfb-0540cad378dd", "content-length": "26", "date": "Wed, 15 Jan 2025 02:34:09 GMT"} })
41912025-01-15T02:34:10.740ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41922025-01-15T02:34:10.742ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41932025-01-15T02:34:10.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = faf77055-bbaa-45f6-a8dc-d9f54de631c7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
41942025-01-15T02:34:10.780ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = faf77055-bbaa-45f6-a8dc-d9f54de631c7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
41952025-01-15T02:34:10.810ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
41962025-01-15T02:34:10.811ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "dde557f2-924b-40bd-8009-5488f8755671", "content-length": "1", "date": "Wed, 15 Jan 2025 02:34:09 GMT"} })
41972025-01-15T02:34:10.812ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
41982025-01-15T02:34:10.812ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41992025-01-15T02:34:10.812ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e9a4d29-b0e0-487c-9cd7-79303852db58", "content-length": "26", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42002025-01-15T02:34:10.812ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42012025-01-15T02:34:10.813ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42022025-01-15T02:34:10.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
42032025-01-15T02:34:10.815ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
42042025-01-15T02:34:10.822ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
42052025-01-15T02:34:10.822ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42062025-01-15T02:34:10.822ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ee050ed-ffe3-4cd8-9f64-3db6aca67f58", "content-length": "372", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42072025-01-15T02:34:10.822ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42082025-01-15T02:34:10.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42092025-01-15T02:34:10.824ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42102025-01-15T02:34:10.824ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44480 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = faf77055-bbaa-45f6-a8dc-d9f54de631c7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
42112025-01-15T02:34:10.824ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
42122025-01-15T02:34:10.824ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
42132025-01-15T02:34:10.824ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
42142025-01-15T02:34:10.824ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 319.735408ms iteration = 8 status = {"generation":1}
42152025-01-15T02:34:10.824ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
42162025-01-15T02:34:10.825ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 320.075247ms iteration = 15 status = null
42172025-01-15T02:34:10.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42182025-01-15T02:34:10.825ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42192025-01-15T02:34:10.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42202025-01-15T02:34:10.827ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
42212025-01-15T02:34:10.827ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "408c0f21-2aa2-4184-a728-6e8a7c0b1d59", "date": "Wed, 15 Jan 2025 02:34:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
42222025-01-15T02:34:10.827ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
42232025-01-15T02:34:10.829ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "70004cd4-ba5f-44af-9a04-97426439b854", "date": "Wed, 15 Jan 2025 02:34:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
42242025-01-15T02:34:10.829ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
42252025-01-15T02:34:10.831ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
42262025-01-15T02:34:10.831ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 326.636105ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
42272025-01-15T02:34:10.866ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
42282025-01-15T02:34:10.866ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
42292025-01-15T02:34:10.866ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "056fe278-3bdd-4afa-adf7-d5fe88d749c2", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42302025-01-15T02:34:10.866ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42312025-01-15T02:34:10.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42322025-01-15T02:34:10.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42332025-01-15T02:34:10.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42342025-01-15T02:34:10.888ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 383.028636ms iteration = 15 status = {}
42352025-01-15T02:34:10.930ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
42362025-01-15T02:34:10.930ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7fcaeef1-56ac-4532-bdcf-bed8f7b56f62", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42372025-01-15T02:34:10.930ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42382025-01-15T02:34:10.932ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
42392025-01-15T02:34:10.932ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5cea51b5-4885-4585-ac55-db63edd01e90", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42402025-01-15T02:34:10.932ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42412025-01-15T02:34:10.932ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2d9140-e05e-4a66-97c9-542b9f91b2f4", "content-length": "691", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42422025-01-15T02:34:10.932ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
42432025-01-15T02:34:10.932ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
42442025-01-15T02:34:10.932ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
42452025-01-15T02:34:10.934ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 428.75647ms iteration = 15 status = {}
42462025-01-15T02:34:10.934ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7904ab3a-f47c-4f4a-8c7b-9d2bd60ae6c7", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42472025-01-15T02:34:10.934ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42482025-01-15T02:34:10.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42492025-01-15T02:34:10.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42502025-01-15T02:34:10.936ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
42512025-01-15T02:34:10.936ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 431.501024ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
42522025-01-15T02:34:10.938ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
42532025-01-15T02:34:10.938ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42542025-01-15T02:34:10.938ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b3df9a02-e047-41ed-874c-5b403cc4ea11", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42552025-01-15T02:34:10.940ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 434.776262ms iteration = 15 status = {}
42562025-01-15T02:34:10.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42572025-01-15T02:34:10.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42582025-01-15T02:34:10.990ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
42592025-01-15T02:34:10.990ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42602025-01-15T02:34:10.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42612025-01-15T02:34:10.998ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
42622025-01-15T02:34:10.999ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b3534f4-724a-495d-9fe8-0af4cfad0d1b", "content-length": "134", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42632025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
42642025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
42652025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
42662025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
42672025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42682025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42692025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42702025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42712025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42722025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42732025-01-15T02:34:10.999ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
42742025-01-15T02:34:10.999ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42752025-01-15T02:34:11.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42762025-01-15T02:34:11.001ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
42772025-01-15T02:34:11.002ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "39d181ad-b4a3-4bef-a678-2a36eb8b0af4", "content-length": "2", "date": "Wed, 15 Jan 2025 02:34:11 GMT"} })
42782025-01-15T02:34:11.002ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
42792025-01-15T02:34:11.002ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8a3bac6-aebc-4594-a76b-57986535d886", "content-length": "52", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42802025-01-15T02:34:11.002ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
42812025-01-15T02:34:11.002ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
42822025-01-15T02:34:11.188ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0360f420-1b3b-4ca2-a35f-af05578942ac", "date": "Wed, 15 Jan 2025 02:34:10 GMT"} })
42832025-01-15T02:34:11.188ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
42842025-01-15T02:34:11.188ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
42852025-01-15T02:34:11.189ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6351e04d-7dde-4971-bf78-f7bf8657086d", "content-length": "682", "date": "Wed, 15 Jan 2025 02:34:11 GMT"} })
42862025-01-15T02:34:11.189ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42872025-01-15T02:34:11.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42882025-01-15T02:34:11.192ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42892025-01-15T02:34:11.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42902025-01-15T02:34:11.195ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42912025-01-15T02:34:11.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42922025-01-15T02:34:11.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42932025-01-15T02:34:11.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42942025-01-15T02:34:11.205ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 699.546507ms iteration = 15 status = {}
42952025-01-15T02:34:11.845ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa00fada-7b38-4501-baf1-6062943d4d94 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
42962025-01-15T02:34:11.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa00fada-7b38-4501-baf1-6062943d4d94 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
42972025-01-15T02:34:11.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1964 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa00fada-7b38-4501-baf1-6062943d4d94 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
42982025-01-15T02:34:12.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7533faa2-496a-4df7-b6d7-645d064f4dbd roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
42992025-01-15T02:34:12.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7533faa2-496a-4df7-b6d7-645d064f4dbd resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43002025-01-15T02:34:12.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2426 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7533faa2-496a-4df7-b6d7-645d064f4dbd response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43012025-01-15T02:34:13.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 105a71e0-81cc-4084-9e5d-7e1a77a52729 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43022025-01-15T02:34:13.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 105a71e0-81cc-4084-9e5d-7e1a77a52729 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43032025-01-15T02:34:13.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2423 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 105a71e0-81cc-4084-9e5d-7e1a77a52729 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43042025-01-15T02:34:14.908ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df655790-d86c-4523-b0a2-221085fc9d4e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43052025-01-15T02:34:14.908ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df655790-d86c-4523-b0a2-221085fc9d4e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43062025-01-15T02:34:14.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2676 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df655790-d86c-4523-b0a2-221085fc9d4e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43072025-01-15T02:34:15.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e5860baf-3821-4394-955c-8ec54b43ae82 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
43082025-01-15T02:34:15.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e5860baf-3821-4394-955c-8ec54b43ae82 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
43092025-01-15T02:34:15.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2923 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e5860baf-3821-4394-955c-8ec54b43ae82 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
43102025-01-15T02:34:15.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e22d0e70-6bad-4809-8bc7-0d64e1377b71 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
43112025-01-15T02:34:15.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e22d0e70-6bad-4809-8bc7-0d64e1377b71 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
43122025-01-15T02:34:15.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e22d0e70-6bad-4809-8bc7-0d64e1377b71 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
43132025-01-15T02:34:15.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 472 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = c82343e0-05b6-4120-8113-1b533cb60708 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
43142025-01-15T02:34:15.931ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8117000b-06f9-447b-b350-8fd49000a66f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43152025-01-15T02:34:15.931ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8117000b-06f9-447b-b350-8fd49000a66f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43162025-01-15T02:34:15.933ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2403 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8117000b-06f9-447b-b350-8fd49000a66f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43172025-01-15T02:34:16.952ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6d7483ee-60ae-4fd1-b21d-13d922bba157 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43182025-01-15T02:34:16.952ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6d7483ee-60ae-4fd1-b21d-13d922bba157 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43192025-01-15T02:34:16.955ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2648 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6d7483ee-60ae-4fd1-b21d-13d922bba157 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43202025-01-15T02:34:17.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23ff0918-10ec-4e52-a669-b8cd5190775f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43212025-01-15T02:34:17.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23ff0918-10ec-4e52-a669-b8cd5190775f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43222025-01-15T02:34:17.976ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1871 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23ff0918-10ec-4e52-a669-b8cd5190775f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43232025-01-15T02:34:18.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8d5f81c0-e604-4ba7-a085-f301fcea3fb1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43242025-01-15T02:34:18.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8d5f81c0-e604-4ba7-a085-f301fcea3fb1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43252025-01-15T02:34:18.999ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8d5f81c0-e604-4ba7-a085-f301fcea3fb1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43262025-01-15T02:34:20.020ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a1320eba-5a6e-4836-a31c-b40ab9fe2b5c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43272025-01-15T02:34:20.020ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a1320eba-5a6e-4836-a31c-b40ab9fe2b5c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43282025-01-15T02:34:20.022ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2223 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a1320eba-5a6e-4836-a31c-b40ab9fe2b5c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43292025-01-15T02:34:20.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
43302025-01-15T02:34:20.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
43312025-01-15T02:34:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43322025-01-15T02:34:20.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43332025-01-15T02:34:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43342025-01-15T02:34:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43352025-01-15T02:34:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43362025-01-15T02:34:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43372025-01-15T02:34:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43382025-01-15T02:34:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43392025-01-15T02:34:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43402025-01-15T02:34:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43412025-01-15T02:34:20.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43422025-01-15T02:34:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43432025-01-15T02:34:20.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43442025-01-15T02:34:20.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
43452025-01-15T02:34:20.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43462025-01-15T02:34:20.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43472025-01-15T02:34:20.556ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
43482025-01-15T02:34:20.556ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.987328ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
43492025-01-15T02:34:20.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.609638ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43502025-01-15T02:34:21.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 31455218-9692-4cf7-bae7-0bfba92b6f10 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43512025-01-15T02:34:21.042ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 31455218-9692-4cf7-bae7-0bfba92b6f10 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43522025-01-15T02:34:21.044ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2340 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 31455218-9692-4cf7-bae7-0bfba92b6f10 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43532025-01-15T02:34:22.065ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 51338009-529e-4315-8fb5-3db69bbb99c9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43542025-01-15T02:34:22.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 51338009-529e-4315-8fb5-3db69bbb99c9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43552025-01-15T02:34:22.066ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1911 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 51338009-529e-4315-8fb5-3db69bbb99c9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43562025-01-15T02:34:23.086ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5b73c7f-fd15-436a-994b-c28c9f77335c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43572025-01-15T02:34:23.087ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5b73c7f-fd15-436a-994b-c28c9f77335c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43582025-01-15T02:34:23.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1972 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5b73c7f-fd15-436a-994b-c28c9f77335c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43592025-01-15T02:34:24.109ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7b776aec-f3bc-4224-ae2a-487eabf0f2cb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43602025-01-15T02:34:24.109ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7b776aec-f3bc-4224-ae2a-487eabf0f2cb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43612025-01-15T02:34:24.111ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2480 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7b776aec-f3bc-4224-ae2a-487eabf0f2cb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43622025-01-15T02:34:25.131ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 673dd652-d12b-4b13-96b0-38ebccd5d462 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43632025-01-15T02:34:25.132ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 673dd652-d12b-4b13-96b0-38ebccd5d462 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43642025-01-15T02:34:25.133ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2360 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 673dd652-d12b-4b13-96b0-38ebccd5d462 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43652025-01-15T02:34:25.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 498 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = e2e0cfa8-81cc-49a2-968f-811c9c63efed response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
43662025-01-15T02:34:26.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 11dc85e8-104e-4874-9215-97796e04eb6b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43672025-01-15T02:34:26.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 11dc85e8-104e-4874-9215-97796e04eb6b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43682025-01-15T02:34:26.157ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 11dc85e8-104e-4874-9215-97796e04eb6b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43692025-01-15T02:34:27.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e886b67-ddd1-44de-b3a4-3e2fea33cdf9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43702025-01-15T02:34:27.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e886b67-ddd1-44de-b3a4-3e2fea33cdf9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43712025-01-15T02:34:27.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2367 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e886b67-ddd1-44de-b3a4-3e2fea33cdf9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43722025-01-15T02:34:28.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc7bbeee-13b6-4920-a861-c106d79cb85c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43732025-01-15T02:34:28.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc7bbeee-13b6-4920-a861-c106d79cb85c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43742025-01-15T02:34:28.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1837 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc7bbeee-13b6-4920-a861-c106d79cb85c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43752025-01-15T02:34:29.221ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9edff7c8-5a08-4ab4-8519-cf695fc231a2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43762025-01-15T02:34:29.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9edff7c8-5a08-4ab4-8519-cf695fc231a2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43772025-01-15T02:34:29.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2452 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9edff7c8-5a08-4ab4-8519-cf695fc231a2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43782025-01-15T02:34:30.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c3fc56d-82ba-40d1-9c05-faa65c663bdb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43792025-01-15T02:34:30.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c3fc56d-82ba-40d1-9c05-faa65c663bdb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43802025-01-15T02:34:30.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3c3fc56d-82ba-40d1-9c05-faa65c663bdb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
43812025-01-15T02:34:30.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
43822025-01-15T02:34:30.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
43832025-01-15T02:34:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43842025-01-15T02:34:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43852025-01-15T02:34:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43862025-01-15T02:34:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43872025-01-15T02:34:30.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43882025-01-15T02:34:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43892025-01-15T02:34:30.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.848693ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43902025-01-15T02:34:30.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43912025-01-15T02:34:30.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43922025-01-15T02:34:30.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43932025-01-15T02:34:30.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43942025-01-15T02:34:30.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43952025-01-15T02:34:30.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43962025-01-15T02:34:30.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43972025-01-15T02:34:30.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
43982025-01-15T02:34:30.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43992025-01-15T02:34:30.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44002025-01-15T02:34:30.560ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
44012025-01-15T02:34:30.560ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 55.534318ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
44022025-01-15T02:34:30.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 34bc1a53-a817-46c0-94c4-44799a92474f roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
44032025-01-15T02:34:30.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 34bc1a53-a817-46c0-94c4-44799a92474f resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
44042025-01-15T02:34:30.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2266 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 34bc1a53-a817-46c0-94c4-44799a92474f response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
44052025-01-15T02:34:30.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 7ea63586-5726-4dc8-b7ee-0daa82c60e2f roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
44062025-01-15T02:34:30.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 7ea63586-5726-4dc8-b7ee-0daa82c60e2f resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
44072025-01-15T02:34:30.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1847 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 7ea63586-5726-4dc8-b7ee-0daa82c60e2f response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
44082025-01-15T02:34:31.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 25b65ce0-1ff6-4af6-89df-f18b7d676dce roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44092025-01-15T02:34:31.267ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 25b65ce0-1ff6-4af6-89df-f18b7d676dce resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44102025-01-15T02:34:31.269ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2193 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 25b65ce0-1ff6-4af6-89df-f18b7d676dce response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44112025-01-15T02:34:32.290ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ff717357-af4e-467f-95c8-e081bd4e79fc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44122025-01-15T02:34:32.290ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ff717357-af4e-467f-95c8-e081bd4e79fc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44132025-01-15T02:34:32.291ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1989 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ff717357-af4e-467f-95c8-e081bd4e79fc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44142025-01-15T02:34:33.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d3153cc0-809f-4cf1-beef-7385efe3b6bc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44152025-01-15T02:34:33.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d3153cc0-809f-4cf1-beef-7385efe3b6bc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44162025-01-15T02:34:33.313ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2185 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d3153cc0-809f-4cf1-beef-7385efe3b6bc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44172025-01-15T02:34:34.333ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 04655cf9-ccc6-4eed-921a-b76a9b8f12f2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44182025-01-15T02:34:34.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 04655cf9-ccc6-4eed-921a-b76a9b8f12f2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44192025-01-15T02:34:34.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1991 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 04655cf9-ccc6-4eed-921a-b76a9b8f12f2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44202025-01-15T02:34:35.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ccfb47f-046a-456b-a793-a20429b16f80 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44212025-01-15T02:34:35.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ccfb47f-046a-456b-a793-a20429b16f80 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44222025-01-15T02:34:35.355ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1542 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1ccfb47f-046a-456b-a793-a20429b16f80 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44232025-01-15T02:34:35.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 448 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = f7a45d72-75d7-488b-b57a-e3e874ca68c7 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
44242025-01-15T02:34:36.376ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c13b21c-2350-4baf-811f-5db2617e137a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44252025-01-15T02:34:36.376ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c13b21c-2350-4baf-811f-5db2617e137a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44262025-01-15T02:34:36.378ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2360 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c13b21c-2350-4baf-811f-5db2617e137a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44272025-01-15T02:34:37.397ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7d487c83-d7bc-4194-a82a-bc1e400c667a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44282025-01-15T02:34:37.398ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7d487c83-d7bc-4194-a82a-bc1e400c667a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44292025-01-15T02:34:37.400ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2776 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7d487c83-d7bc-4194-a82a-bc1e400c667a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44302025-01-15T02:34:38.419ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e967c2f-8009-4217-8a15-499791693503 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44312025-01-15T02:34:38.419ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e967c2f-8009-4217-8a15-499791693503 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44322025-01-15T02:34:38.421ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1855 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e967c2f-8009-4217-8a15-499791693503 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44332025-01-15T02:34:39.441ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 919af42c-76ae-4f38-8497-fde687ed9aa0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44342025-01-15T02:34:39.441ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 919af42c-76ae-4f38-8497-fde687ed9aa0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44352025-01-15T02:34:39.443ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2100 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 919af42c-76ae-4f38-8497-fde687ed9aa0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44362025-01-15T02:34:40.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fdbb9f0c-2f7b-404c-8e37-5b20cbeeb2a7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44372025-01-15T02:34:40.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fdbb9f0c-2f7b-404c-8e37-5b20cbeeb2a7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44382025-01-15T02:34:40.465ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2116 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fdbb9f0c-2f7b-404c-8e37-5b20cbeeb2a7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
44392025-01-15T02:34:40.504ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
44402025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
44412025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
44422025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
44432025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
44442025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
44452025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
44462025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
44472025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
44482025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
44492025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44502025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44512025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
44522025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44532025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44542025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
44552025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44562025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
44572025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
44582025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44592025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
44602025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
44612025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44622025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44632025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
44642025-01-15T02:34:40.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
44652025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44662025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
44672025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44682025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
44692025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44702025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44712025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44722025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44732025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44742025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
44752025-01-15T02:34:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
44762025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44772025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44782025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44792025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44802025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44812025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
44822025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
44832025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
44842025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
44852025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44862025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44872025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
44882025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
44892025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
44902025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44912025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
44922025-01-15T02:34:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44932025-01-15T02:34:40.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3767, tv_nsec: 344298975 } })
44942025-01-15T02:34:40.508ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
44952025-01-15T02:34:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44962025-01-15T02:34:40.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3767, tv_nsec: 345590670 } })
44972025-01-15T02:34:40.509ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44982025-01-15T02:34:40.509ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.318186ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
44992025-01-15T02:34:40.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
45002025-01-15T02:34:40.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45012025-01-15T02:34:40.509ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45022025-01-15T02:34:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45032025-01-15T02:34:40.511ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45042025-01-15T02:34:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45052025-01-15T02:34:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
45062025-01-15T02:34:40.512ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45072025-01-15T02:34:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45082025-01-15T02:34:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
45092025-01-15T02:34:40.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45102025-01-15T02:34:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45112025-01-15T02:34:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45122025-01-15T02:34:40.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45132025-01-15T02:34:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45142025-01-15T02:34:40.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45152025-01-15T02:34:40.517ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45162025-01-15T02:34:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45172025-01-15T02:34:40.518ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45182025-01-15T02:34:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
45192025-01-15T02:34:40.518ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
45202025-01-15T02:34:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
45212025-01-15T02:34:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
45222025-01-15T02:34:40.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45232025-01-15T02:34:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45242025-01-15T02:34:40.520ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45252025-01-15T02:34:40.520ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
45262025-01-15T02:34:40.520ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
45272025-01-15T02:34:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
45282025-01-15T02:34:40.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45292025-01-15T02:34:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45302025-01-15T02:34:40.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45312025-01-15T02:34:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45322025-01-15T02:34:40.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45332025-01-15T02:34:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45342025-01-15T02:34:40.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
45352025-01-15T02:34:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
45362025-01-15T02:34:40.539ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3767, tv_nsec: 376216316 } })
45372025-01-15T02:34:40.540ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
45382025-01-15T02:34:40.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
45392025-01-15T02:34:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
45402025-01-15T02:34:40.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45412025-01-15T02:34:40.567ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45422025-01-15T02:34:40.567ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45432025-01-15T02:34:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45442025-01-15T02:34:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45452025-01-15T02:34:40.568ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
45462025-01-15T02:34:40.568ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.300818ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
45472025-01-15T02:34:40.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.046451ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
45482025-01-15T02:34:40.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45492025-01-15T02:34:40.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45502025-01-15T02:34:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45512025-01-15T02:34:40.570ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45522025-01-15T02:34:40.571ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45532025-01-15T02:34:40.571ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
45542025-01-15T02:34:40.571ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
45552025-01-15T02:34:40.571ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45562025-01-15T02:34:40.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45572025-01-15T02:34:40.571ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef2b63bd-3378-4f22-adf9-8e4373602d66", "content-length": "26", "date": "Wed, 15 Jan 2025 02:34:39 GMT"} })
45582025-01-15T02:34:40.571ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
45592025-01-15T02:34:40.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45602025-01-15T02:34:40.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45612025-01-15T02:34:40.572ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45622025-01-15T02:34:40.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45632025-01-15T02:34:40.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45642025-01-15T02:34:40.573ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.143848ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45652025-01-15T02:34:40.574ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
45662025-01-15T02:34:40.574ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
45672025-01-15T02:34:40.574ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
45682025-01-15T02:34:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.747929ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
45692025-01-15T02:34:40.574ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbf829d7-c90c-478f-91e7-66e588d56b5c", "content-length": "26", "date": "Wed, 15 Jan 2025 02:34:39 GMT"} })
45702025-01-15T02:34:40.574ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
45712025-01-15T02:34:40.575ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45722025-01-15T02:34:40.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45732025-01-15T02:34:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45742025-01-15T02:34:40.576ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
45752025-01-15T02:34:40.580ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45762025-01-15T02:34:40.580ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
45772025-01-15T02:34:40.580ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
45782025-01-15T02:34:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
45792025-01-15T02:34:40.582ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 75.61521ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
45802025-01-15T02:34:40.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45812025-01-15T02:34:40.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
45822025-01-15T02:34:40.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45832025-01-15T02:34:40.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45842025-01-15T02:34:40.606ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
45852025-01-15T02:34:40.606ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 100.702318ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
45862025-01-15T02:34:40.610ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 104.666935ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
45872025-01-15T02:34:40.612ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45882025-01-15T02:34:40.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45892025-01-15T02:34:40.614ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45902025-01-15T02:34:40.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
45912025-01-15T02:34:40.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45922025-01-15T02:34:40.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45932025-01-15T02:34:40.618ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45942025-01-15T02:34:40.621ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45952025-01-15T02:34:40.621ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
45962025-01-15T02:34:40.621ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 116.417171ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
45972025-01-15T02:34:40.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
45982025-01-15T02:34:40.623ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45992025-01-15T02:34:40.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46002025-01-15T02:34:40.625ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.894485ms iteration = 16 status = 0
46012025-01-15T02:34:40.626ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
46022025-01-15T02:34:40.626ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46032025-01-15T02:34:40.627ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbd812a1-0202-4d34-9847-b5a7a73c7042", "content-length": "26", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46042025-01-15T02:34:40.627ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46052025-01-15T02:34:40.628ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
46062025-01-15T02:34:40.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 146.572076ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
46072025-01-15T02:34:40.659ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46082025-01-15T02:34:40.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46092025-01-15T02:34:40.671ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
46102025-01-15T02:34:40.671ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 166.271586ms iteration = 16 status = {}
46112025-01-15T02:34:40.671ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "18489647-5850-4970-8ff3-ea4197f542f7", "content-length": "1", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46122025-01-15T02:34:40.672ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
46132025-01-15T02:34:40.672ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 166.824591ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
46142025-01-15T02:34:40.673ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
46152025-01-15T02:34:40.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
46162025-01-15T02:34:40.692ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 186.704608ms iteration = 18 status = {"physical_disks_added":0}
46172025-01-15T02:34:40.706ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
46182025-01-15T02:34:40.706ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
46192025-01-15T02:34:40.707ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d5beff6-806f-46f3-8e7f-a0e94a2f7f60", "content-length": "372", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46202025-01-15T02:34:40.707ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46212025-01-15T02:34:40.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46222025-01-15T02:34:40.709ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 204.295853ms iteration = 16 status = null
46232025-01-15T02:34:40.733ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
46242025-01-15T02:34:40.733ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
46252025-01-15T02:34:40.734ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8c2d060e-5e33-41d9-9ad6-474f5194bfb8", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46262025-01-15T02:34:40.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46272025-01-15T02:34:40.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46282025-01-15T02:34:40.737ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46292025-01-15T02:34:40.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46302025-01-15T02:34:40.742ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46312025-01-15T02:34:40.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46322025-01-15T02:34:40.746ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
46332025-01-15T02:34:40.746ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
46342025-01-15T02:34:40.746ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3384aa06-05d0-443e-8843-7e7171dad998", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46352025-01-15T02:34:40.749ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 243.486191ms iteration = 16 status = {}
46362025-01-15T02:34:40.777ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46372025-01-15T02:34:40.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46382025-01-15T02:34:40.780ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
46392025-01-15T02:34:40.780ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46402025-01-15T02:34:40.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46412025-01-15T02:34:40.792ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
46422025-01-15T02:34:40.793ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f202d03-bb88-484a-8f4e-91e22f5e44f2", "content-length": "134", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46432025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
46442025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
46452025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
46462025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
46472025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46482025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46492025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46502025-01-15T02:34:40.793ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
46512025-01-15T02:34:40.794ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
46522025-01-15T02:34:40.794ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
46532025-01-15T02:34:40.794ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
46542025-01-15T02:34:40.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46552025-01-15T02:34:40.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46562025-01-15T02:34:40.797ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
46572025-01-15T02:34:40.797ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "223adc90-9aaf-4ab3-ae41-3bcb657a997a", "content-length": "2", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46582025-01-15T02:34:40.797ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
46592025-01-15T02:34:40.797ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c3ae7d3-3404-4b5b-84b3-8a53658e0239", "content-length": "52", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46602025-01-15T02:34:40.797ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
46612025-01-15T02:34:40.797ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
46622025-01-15T02:34:40.827ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46632025-01-15T02:34:40.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46642025-01-15T02:34:40.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46652025-01-15T02:34:40.832ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
46662025-01-15T02:34:40.832ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46672025-01-15T02:34:40.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46682025-01-15T02:34:40.834ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46692025-01-15T02:34:40.836ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
46702025-01-15T02:34:40.836ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46712025-01-15T02:34:40.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46722025-01-15T02:34:40.936ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d18f4f6b-ce7f-46e1-9e0f-7563a288f2b9", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46732025-01-15T02:34:40.936ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
46742025-01-15T02:34:40.936ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
46752025-01-15T02:34:40.937ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "00a67b2a-60cb-48b8-b09a-754b00cc8817", "content-length": "682", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46762025-01-15T02:34:40.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46772025-01-15T02:34:40.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46782025-01-15T02:34:40.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46792025-01-15T02:34:40.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46802025-01-15T02:34:40.944ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46812025-01-15T02:34:40.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46822025-01-15T02:34:40.946ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46832025-01-15T02:34:40.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46842025-01-15T02:34:40.950ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
46852025-01-15T02:34:40.950ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ea7e2f6-98b3-412a-8d70-06e89e8a3bbf", "content-length": "691", "date": "Wed, 15 Jan 2025 02:34:40 GMT"} })
46862025-01-15T02:34:40.950ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
46872025-01-15T02:34:40.950ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
46882025-01-15T02:34:40.950ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
46892025-01-15T02:34:40.952ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 446.707464ms iteration = 16 status = {}
46902025-01-15T02:34:40.954ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 448.893093ms iteration = 16 status = {}
46912025-01-15T02:34:41.488ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 57112195-d728-4d17-89c3-b207d3cc8def roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46922025-01-15T02:34:41.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 57112195-d728-4d17-89c3-b207d3cc8def resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46932025-01-15T02:34:41.490ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1687 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 57112195-d728-4d17-89c3-b207d3cc8def response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46942025-01-15T02:34:42.510ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5014f764-4723-4f57-aa9f-e8e0ae2b66ec roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46952025-01-15T02:34:42.510ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5014f764-4723-4f57-aa9f-e8e0ae2b66ec resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46962025-01-15T02:34:42.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1987 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5014f764-4723-4f57-aa9f-e8e0ae2b66ec response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46972025-01-15T02:34:43.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2c22ae17-6e70-4e7f-9a9c-a8299568518d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46982025-01-15T02:34:43.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2c22ae17-6e70-4e7f-9a9c-a8299568518d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
46992025-01-15T02:34:43.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2093 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2c22ae17-6e70-4e7f-9a9c-a8299568518d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47002025-01-15T02:34:44.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 002b6e98-61d3-416f-90ab-907d1026b88d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47012025-01-15T02:34:44.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 002b6e98-61d3-416f-90ab-907d1026b88d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47022025-01-15T02:34:44.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1821 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 002b6e98-61d3-416f-90ab-907d1026b88d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47032025-01-15T02:34:45.574ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8f76509f-8091-40d6-91f7-b12ea0017dc7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47042025-01-15T02:34:45.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8f76509f-8091-40d6-91f7-b12ea0017dc7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47052025-01-15T02:34:45.576ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2001 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8f76509f-8091-40d6-91f7-b12ea0017dc7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47062025-01-15T02:34:45.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 444 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 570c5600-2779-4777-a332-d2c806236338 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
47072025-01-15T02:34:46.596ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f958ccaa-7f2b-4a20-a636-f96359d2091d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47082025-01-15T02:34:46.596ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f958ccaa-7f2b-4a20-a636-f96359d2091d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47092025-01-15T02:34:46.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1559 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f958ccaa-7f2b-4a20-a636-f96359d2091d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47102025-01-15T02:34:47.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c9b9bc9-6082-45da-a2ae-967b1e0a03d7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47112025-01-15T02:34:47.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c9b9bc9-6082-45da-a2ae-967b1e0a03d7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47122025-01-15T02:34:47.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1905 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c9b9bc9-6082-45da-a2ae-967b1e0a03d7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47132025-01-15T02:34:48.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 941d2cbd-b81c-46c8-8cf9-4e9b2b973f7b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47142025-01-15T02:34:48.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 941d2cbd-b81c-46c8-8cf9-4e9b2b973f7b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47152025-01-15T02:34:48.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2244 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 941d2cbd-b81c-46c8-8cf9-4e9b2b973f7b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47162025-01-15T02:34:49.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4f913e28-90d8-4384-a3c9-f7c29008437c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47172025-01-15T02:34:49.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4f913e28-90d8-4384-a3c9-f7c29008437c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47182025-01-15T02:34:49.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1750 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4f913e28-90d8-4384-a3c9-f7c29008437c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47192025-01-15T02:34:50.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
47202025-01-15T02:34:50.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
47212025-01-15T02:34:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47222025-01-15T02:34:50.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47232025-01-15T02:34:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47242025-01-15T02:34:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47252025-01-15T02:34:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47262025-01-15T02:34:50.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47272025-01-15T02:34:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47282025-01-15T02:34:50.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.174633ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47292025-01-15T02:34:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47302025-01-15T02:34:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47312025-01-15T02:34:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47322025-01-15T02:34:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47332025-01-15T02:34:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47342025-01-15T02:34:50.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47352025-01-15T02:34:50.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
47362025-01-15T02:34:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47372025-01-15T02:34:50.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47382025-01-15T02:34:50.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
47392025-01-15T02:34:50.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.586353ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
47402025-01-15T02:34:50.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d02f7ef9-d795-4e66-97a0-95249001e72b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47412025-01-15T02:34:50.689ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d02f7ef9-d795-4e66-97a0-95249001e72b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47422025-01-15T02:34:50.690ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1663 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d02f7ef9-d795-4e66-97a0-95249001e72b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47432025-01-15T02:34:51.712ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82724a02-0975-4606-991a-e4e3c4839bc6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47442025-01-15T02:34:51.712ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82724a02-0975-4606-991a-e4e3c4839bc6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47452025-01-15T02:34:51.713ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1863 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82724a02-0975-4606-991a-e4e3c4839bc6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47462025-01-15T02:34:52.733ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 93dcb616-b8da-4ba7-853d-6c8e0fd9c91d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47472025-01-15T02:34:52.734ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 93dcb616-b8da-4ba7-853d-6c8e0fd9c91d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47482025-01-15T02:34:52.735ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1996 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 93dcb616-b8da-4ba7-853d-6c8e0fd9c91d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47492025-01-15T02:34:53.754ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e947b6bd-7789-4267-b475-29d3ebf75f94 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47502025-01-15T02:34:53.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e947b6bd-7789-4267-b475-29d3ebf75f94 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47512025-01-15T02:34:53.756ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e947b6bd-7789-4267-b475-29d3ebf75f94 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47522025-01-15T02:34:54.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f2f3d434-d263-40e2-a453-ff8a30acfa6d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47532025-01-15T02:34:54.777ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f2f3d434-d263-40e2-a453-ff8a30acfa6d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47542025-01-15T02:34:54.779ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2541 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f2f3d434-d263-40e2-a453-ff8a30acfa6d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47552025-01-15T02:34:55.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 508 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = dca50063-2c52-4ebf-b3d6-199f31cc5a51 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
47562025-01-15T02:34:55.798ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 469ec5f4-5fff-4fa3-9b2c-6dc9f442d6e0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47572025-01-15T02:34:55.798ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 469ec5f4-5fff-4fa3-9b2c-6dc9f442d6e0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47582025-01-15T02:34:55.800ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1889 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 469ec5f4-5fff-4fa3-9b2c-6dc9f442d6e0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47592025-01-15T02:34:56.821ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 229c19df-a6c9-4c92-ae02-5b7f44ccea06 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47602025-01-15T02:34:56.821ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 229c19df-a6c9-4c92-ae02-5b7f44ccea06 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47612025-01-15T02:34:56.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2225 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 229c19df-a6c9-4c92-ae02-5b7f44ccea06 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47622025-01-15T02:34:57.842ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d57a1834-1853-47a4-a190-a0cd1239ad81 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47632025-01-15T02:34:57.843ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d57a1834-1853-47a4-a190-a0cd1239ad81 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47642025-01-15T02:34:57.844ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1798 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d57a1834-1853-47a4-a190-a0cd1239ad81 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47652025-01-15T02:34:58.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 644555ab-c508-47fa-92b7-04111e1bd9ab roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47662025-01-15T02:34:58.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 644555ab-c508-47fa-92b7-04111e1bd9ab resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47672025-01-15T02:34:58.868ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2721 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 644555ab-c508-47fa-92b7-04111e1bd9ab response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47682025-01-15T02:34:59.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cba42cb-f86c-4691-9c0f-5a6201c5e690 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47692025-01-15T02:34:59.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cba42cb-f86c-4691-9c0f-5a6201c5e690 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47702025-01-15T02:34:59.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2698 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cba42cb-f86c-4691-9c0f-5a6201c5e690 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47712025-01-15T02:35:00.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
47722025-01-15T02:35:00.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
47732025-01-15T02:35:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47742025-01-15T02:35:00.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47752025-01-15T02:35:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47762025-01-15T02:35:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47772025-01-15T02:35:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47782025-01-15T02:35:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47792025-01-15T02:35:00.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.667229ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47802025-01-15T02:35:00.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47812025-01-15T02:35:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47822025-01-15T02:35:00.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47832025-01-15T02:35:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47842025-01-15T02:35:00.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47852025-01-15T02:35:00.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47862025-01-15T02:35:00.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47872025-01-15T02:35:00.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
47882025-01-15T02:35:00.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47892025-01-15T02:35:00.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47902025-01-15T02:35:00.554ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
47912025-01-15T02:35:00.554ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.362252ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
47922025-01-15T02:35:00.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4b16f9c-46c6-4299-8a27-9e50a4174abb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47932025-01-15T02:35:00.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4b16f9c-46c6-4299-8a27-9e50a4174abb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47942025-01-15T02:35:00.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2152 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4b16f9c-46c6-4299-8a27-9e50a4174abb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47952025-01-15T02:35:01.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a56088b4-2930-4c37-bc98-ad45bb3c0bfa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47962025-01-15T02:35:01.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a56088b4-2930-4c37-bc98-ad45bb3c0bfa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47972025-01-15T02:35:01.936ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1689 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a56088b4-2930-4c37-bc98-ad45bb3c0bfa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47982025-01-15T02:35:02.956ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fc6d0917-ff99-47b7-b83e-6e2f5822e8bb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
47992025-01-15T02:35:02.956ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fc6d0917-ff99-47b7-b83e-6e2f5822e8bb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48002025-01-15T02:35:02.958ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2024 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fc6d0917-ff99-47b7-b83e-6e2f5822e8bb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48012025-01-15T02:35:03.977ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b60e9328-3ff3-4901-b6f4-6f150dc4f216 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48022025-01-15T02:35:03.977ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b60e9328-3ff3-4901-b6f4-6f150dc4f216 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48032025-01-15T02:35:03.979ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2028 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b60e9328-3ff3-4901-b6f4-6f150dc4f216 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48042025-01-15T02:35:04.998ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32ef5de5-0f1e-4030-8a58-c6bd4e398a31 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48052025-01-15T02:35:04.999ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32ef5de5-0f1e-4030-8a58-c6bd4e398a31 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48062025-01-15T02:35:05.000ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1878 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32ef5de5-0f1e-4030-8a58-c6bd4e398a31 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48072025-01-15T02:35:05.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 479 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 8fd2e2fc-2fc2-458f-b9fb-19aa378f3feb response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
48082025-01-15T02:35:06.020ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fb48b062-080a-47d5-9413-ed4f486df65a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48092025-01-15T02:35:06.020ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fb48b062-080a-47d5-9413-ed4f486df65a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48102025-01-15T02:35:06.022ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1890 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fb48b062-080a-47d5-9413-ed4f486df65a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48112025-01-15T02:35:07.043ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6e2a9064-d10a-40eb-ac18-264c3c8220f0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48122025-01-15T02:35:07.043ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6e2a9064-d10a-40eb-ac18-264c3c8220f0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48132025-01-15T02:35:07.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2318 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6e2a9064-d10a-40eb-ac18-264c3c8220f0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48142025-01-15T02:35:08.066ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8553fecf-4ffe-4cbb-9530-3dc30ad33e86 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48152025-01-15T02:35:08.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8553fecf-4ffe-4cbb-9530-3dc30ad33e86 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48162025-01-15T02:35:08.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2655 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8553fecf-4ffe-4cbb-9530-3dc30ad33e86 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48172025-01-15T02:35:09.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb2c8b1b-9b10-4311-a06b-2ed047d697a0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48182025-01-15T02:35:09.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb2c8b1b-9b10-4311-a06b-2ed047d697a0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48192025-01-15T02:35:09.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2185 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb2c8b1b-9b10-4311-a06b-2ed047d697a0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48202025-01-15T02:35:10.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b6d19c6a-980b-467e-8fdd-64f6be14702f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48212025-01-15T02:35:10.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b6d19c6a-980b-467e-8fdd-64f6be14702f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48222025-01-15T02:35:10.114ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1922 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b6d19c6a-980b-467e-8fdd-64f6be14702f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
48232025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
48242025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48252025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
48262025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
48272025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
48282025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48292025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
48302025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
48312025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
48322025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48332025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
48342025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
48352025-01-15T02:35:10.504ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:25:10.504815589 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
48362025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
48372025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
48382025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
48392025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
48402025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
48412025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
48422025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
48432025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
48442025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
48452025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48462025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48472025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
48482025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
48492025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48502025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
48512025-01-15T02:35:10.504ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
48522025-01-15T02:35:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
48532025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
48542025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
48552025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Timeout
48562025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
48572025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48582025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
48592025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48602025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48612025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
48622025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
48632025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48642025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
48652025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48662025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48672025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
48682025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
48692025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
48702025-01-15T02:35:10.505ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
48712025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.24\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
48722025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
48732025-01-15T02:35:10.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
48742025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48752025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48762025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48772025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
48782025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48792025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
48802025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
48812025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48822025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48832025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48842025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
48852025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
48862025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48872025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48882025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48892025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
48902025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48912025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48922025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48932025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48942025-01-15T02:35:10.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3797, tv_nsec: 342251310 } })
48952025-01-15T02:35:10.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3797, tv_nsec: 342258740 } })
48962025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
48972025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48982025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
48992025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
49002025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
49012025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
49022025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
49032025-01-15T02:35:10.505ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
49042025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
49052025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
49062025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
49072025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
49082025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
49092025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
49102025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49112025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
49122025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.48157ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
49132025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
49142025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
49152025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
49162025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
49172025-01-15T02:35:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.408318ms iteration = 9 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
49182025-01-15T02:35:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
49192025-01-15T02:35:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
49202025-01-15T02:35:10.509ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
49212025-01-15T02:35:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
49222025-01-15T02:35:10.510ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3797, tv_nsec: 346622098 } })
49232025-01-15T02:35:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.168009ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
49242025-01-15T02:35:10.510ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
49252025-01-15T02:35:10.510ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:25:10.504815589 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
49262025-01-15T02:35:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.796646ms iteration = 9 status = {"expiration":"2025-01-15T02:25:10.504815589Z","pruned":[]}
49272025-01-15T02:35:10.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49282025-01-15T02:35:10.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49292025-01-15T02:35:10.512ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3797, tv_nsec: 349226768 } })
49302025-01-15T02:35:10.512ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
49312025-01-15T02:35:10.512ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
49322025-01-15T02:35:10.512ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
49332025-01-15T02:35:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
49342025-01-15T02:35:10.514ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49352025-01-15T02:35:10.514ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
49362025-01-15T02:35:10.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49372025-01-15T02:35:10.514ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49382025-01-15T02:35:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49392025-01-15T02:35:10.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49402025-01-15T02:35:10.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49412025-01-15T02:35:10.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49422025-01-15T02:35:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
49432025-01-15T02:35:10.517ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
49442025-01-15T02:35:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
49452025-01-15T02:35:10.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49462025-01-15T02:35:10.518ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49472025-01-15T02:35:10.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49482025-01-15T02:35:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49492025-01-15T02:35:10.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49502025-01-15T02:35:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49512025-01-15T02:35:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
49522025-01-15T02:35:10.521ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
49532025-01-15T02:35:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
49542025-01-15T02:35:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
49552025-01-15T02:35:10.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49562025-01-15T02:35:10.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49572025-01-15T02:35:10.522ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49582025-01-15T02:35:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49592025-01-15T02:35:10.524ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49602025-01-15T02:35:10.524ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49612025-01-15T02:35:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49622025-01-15T02:35:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
49632025-01-15T02:35:10.525ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49642025-01-15T02:35:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49652025-01-15T02:35:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 19.502297ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
49662025-01-15T02:35:10.526ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 20.512404ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
49672025-01-15T02:35:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.045018ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
49682025-01-15T02:35:10.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49692025-01-15T02:35:10.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
49702025-01-15T02:35:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
49712025-01-15T02:35:10.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49722025-01-15T02:35:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49732025-01-15T02:35:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
49742025-01-15T02:35:10.533ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
49752025-01-15T02:35:10.533ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
49762025-01-15T02:35:10.533ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
49772025-01-15T02:35:10.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
49782025-01-15T02:35:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
49792025-01-15T02:35:10.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49802025-01-15T02:35:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49812025-01-15T02:35:10.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49822025-01-15T02:35:10.535ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49832025-01-15T02:35:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49842025-01-15T02:35:10.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49852025-01-15T02:35:10.537ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49862025-01-15T02:35:10.537ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49872025-01-15T02:35:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49882025-01-15T02:35:10.539ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
49892025-01-15T02:35:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 34.409809ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
49902025-01-15T02:35:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
49912025-01-15T02:35:10.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49922025-01-15T02:35:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49932025-01-15T02:35:10.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49942025-01-15T02:35:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49952025-01-15T02:35:10.550ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49962025-01-15T02:35:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49972025-01-15T02:35:10.565ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49982025-01-15T02:35:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49992025-01-15T02:35:10.567ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
50002025-01-15T02:35:10.567ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 62.792134ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
50012025-01-15T02:35:10.568ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50022025-01-15T02:35:10.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50032025-01-15T02:35:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50042025-01-15T02:35:10.572ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
50052025-01-15T02:35:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
50062025-01-15T02:35:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
50072025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50082025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50092025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
50102025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
50112025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
50122025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50132025-01-15T02:35:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50142025-01-15T02:35:10.575ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50152025-01-15T02:35:10.576ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
50162025-01-15T02:35:10.576ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
50172025-01-15T02:35:10.576ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 70.441881ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
50182025-01-15T02:35:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
50192025-01-15T02:35:10.577ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50202025-01-15T02:35:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50212025-01-15T02:35:10.578ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50222025-01-15T02:35:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50232025-01-15T02:35:10.581ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
50242025-01-15T02:35:10.581ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
50252025-01-15T02:35:10.581ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50262025-01-15T02:35:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
50272025-01-15T02:35:10.586ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
50282025-01-15T02:35:10.586ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
50292025-01-15T02:35:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 81.703084ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
50302025-01-15T02:35:10.587ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50312025-01-15T02:35:10.588ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50322025-01-15T02:35:10.589ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50332025-01-15T02:35:10.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50342025-01-15T02:35:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
50352025-01-15T02:35:10.591ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50362025-01-15T02:35:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50372025-01-15T02:35:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
50382025-01-15T02:35:10.592ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50392025-01-15T02:35:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50402025-01-15T02:35:10.595ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
50412025-01-15T02:35:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
50422025-01-15T02:35:10.620ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
50432025-01-15T02:35:10.628ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
50442025-01-15T02:35:10.634ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50452025-01-15T02:35:10.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50462025-01-15T02:35:10.644ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
50472025-01-15T02:35:10.644ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50482025-01-15T02:35:10.644ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "30652833-6640-43c9-87b3-89d57d703179", "content-length": "26", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
50492025-01-15T02:35:10.644ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50502025-01-15T02:35:10.646ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50512025-01-15T02:35:10.646ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
50522025-01-15T02:35:10.646ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50532025-01-15T02:35:10.646ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c48e7cd2-06ef-4565-93dc-63fd66ae834a", "content-length": "26", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
50542025-01-15T02:35:10.646ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50552025-01-15T02:35:10.648ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50562025-01-15T02:35:10.671ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
50572025-01-15T02:35:10.671ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50582025-01-15T02:35:10.672ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3847dc51-9dee-4c61-bb9e-97a3384d948a", "content-length": "372", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
50592025-01-15T02:35:10.672ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50602025-01-15T02:35:10.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50612025-01-15T02:35:10.673ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 168.509087ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50622025-01-15T02:35:10.675ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50632025-01-15T02:35:10.675ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50642025-01-15T02:35:10.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50652025-01-15T02:35:10.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50662025-01-15T02:35:10.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50672025-01-15T02:35:10.677ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3797, tv_nsec: 513786799 } })
50682025-01-15T02:35:10.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50692025-01-15T02:35:10.677ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
50702025-01-15T02:35:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
50712025-01-15T02:35:10.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50722025-01-15T02:35:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50732025-01-15T02:35:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
50742025-01-15T02:35:10.679ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50752025-01-15T02:35:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50762025-01-15T02:35:10.711ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
50772025-01-15T02:35:10.711ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
50782025-01-15T02:35:10.711ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50792025-01-15T02:35:10.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50802025-01-15T02:35:10.713ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 207.940142ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
50812025-01-15T02:35:10.716ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 211.191969ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
50822025-01-15T02:35:10.720ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
50832025-01-15T02:35:10.720ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
50842025-01-15T02:35:10.720ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 215.31382ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
50852025-01-15T02:35:10.722ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
50862025-01-15T02:35:10.722ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
50872025-01-15T02:35:10.722ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50882025-01-15T02:35:10.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50892025-01-15T02:35:10.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50902025-01-15T02:35:10.726ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50912025-01-15T02:35:10.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50922025-01-15T02:35:10.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
50932025-01-15T02:35:10.727ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50942025-01-15T02:35:10.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50952025-01-15T02:35:10.727ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 222.095353ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
50962025-01-15T02:35:10.728ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50972025-01-15T02:35:10.729ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
50982025-01-15T02:35:10.729ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
50992025-01-15T02:35:10.729ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
51002025-01-15T02:35:10.729ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 225.217288ms iteration = 9 status = {"generation":2}
51012025-01-15T02:35:10.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
51022025-01-15T02:35:10.729ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51032025-01-15T02:35:10.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
51042025-01-15T02:35:10.730ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51052025-01-15T02:35:10.731ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
51062025-01-15T02:35:10.731ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 226.205784ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
51072025-01-15T02:35:10.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
51082025-01-15T02:35:10.732ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51092025-01-15T02:35:10.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51102025-01-15T02:35:10.734ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 229.662227ms iteration = 17 status = 0
51112025-01-15T02:35:10.744ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
51122025-01-15T02:35:10.744ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
51132025-01-15T02:35:10.744ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7e57a3e0-5e39-4b76-9e25-d2fb8b55f60d", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
51142025-01-15T02:35:10.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51152025-01-15T02:35:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51162025-01-15T02:35:10.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51172025-01-15T02:35:10.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51182025-01-15T02:35:10.749ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51192025-01-15T02:35:10.750ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51202025-01-15T02:35:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51212025-01-15T02:35:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
51222025-01-15T02:35:10.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51232025-01-15T02:35:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51242025-01-15T02:35:10.754ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
51252025-01-15T02:35:10.754ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
51262025-01-15T02:35:10.754ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
51272025-01-15T02:35:10.754ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 249.296068ms iteration = 9 status = {"generation":1}
51282025-01-15T02:35:10.754ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
51292025-01-15T02:35:10.754ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
51302025-01-15T02:35:10.755ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b598d6e3-6320-4213-943c-a7c0d26d56d2", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
51312025-01-15T02:35:10.756ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 251.361322ms iteration = 17 status = {}
51322025-01-15T02:35:10.767ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
51332025-01-15T02:35:10.770ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
51342025-01-15T02:35:10.770ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 266.08421ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
51352025-01-15T02:35:10.778ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
51362025-01-15T02:35:10.778ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 273.171711ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
51372025-01-15T02:35:10.797ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
51382025-01-15T02:35:10.798ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6819a8f1-ba7e-4677-bd90-505bc5f693a1", "content-length": "1", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
51392025-01-15T02:35:10.799ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
51402025-01-15T02:35:10.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
51412025-01-15T02:35:10.801ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 296.382718ms iteration = 19 status = {"physical_disks_added":0}
51422025-01-15T02:35:10.801ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 296.980634ms iteration = 17 status = null
51432025-01-15T02:35:10.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51442025-01-15T02:35:10.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51452025-01-15T02:35:10.816ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
51462025-01-15T02:35:10.816ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51472025-01-15T02:35:10.816ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "caebd310-7b36-493e-bc26-84a148cfb938", "content-length": "26", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
51482025-01-15T02:35:10.816ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51492025-01-15T02:35:10.817ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
51502025-01-15T02:35:10.837ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51512025-01-15T02:35:10.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
51522025-01-15T02:35:10.838ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51532025-01-15T02:35:10.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51542025-01-15T02:35:10.840ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51552025-01-15T02:35:10.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
51562025-01-15T02:35:10.841ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51572025-01-15T02:35:10.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51582025-01-15T02:35:10.882ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
51592025-01-15T02:35:10.883ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c58e3d95-0119-4a01-a46a-9b596240df94", "date": "Wed, 15 Jan 2025 02:35:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
51602025-01-15T02:35:10.883ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
51612025-01-15T02:35:10.885ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "241c032d-8531-43f8-9ff0-27f42e6af326", "date": "Wed, 15 Jan 2025 02:35:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
51622025-01-15T02:35:10.885ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
51632025-01-15T02:35:10.886ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
51642025-01-15T02:35:10.886ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 381.592377ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
51652025-01-15T02:35:10.892ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
51662025-01-15T02:35:10.892ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 387.479176ms iteration = 17 status = {}
51672025-01-15T02:35:10.892ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "178b9f3b-525c-471c-ae5e-c2a79c931cb5", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51682025-01-15T02:35:10.892ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51692025-01-15T02:35:10.894ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c1112e7b-62fb-4282-8beb-f9e868274930", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51702025-01-15T02:35:10.894ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51712025-01-15T02:35:10.895ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "57254f15-f496-42d6-abe3-be6de3a79437", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51722025-01-15T02:35:10.895ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51732025-01-15T02:35:10.896ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
51742025-01-15T02:35:10.896ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 391.999358ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
51752025-01-15T02:35:10.958ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
51762025-01-15T02:35:10.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51772025-01-15T02:35:10.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51782025-01-15T02:35:10.991ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
51792025-01-15T02:35:10.991ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51802025-01-15T02:35:10.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51812025-01-15T02:35:11.001ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
51822025-01-15T02:35:11.001ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3920f88-b167-4749-b2c7-b9d3c4513e3d", "content-length": "134", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
51832025-01-15T02:35:11.001ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
51842025-01-15T02:35:11.001ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
51852025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
51862025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
51872025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51882025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51892025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51902025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
51912025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51922025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
51932025-01-15T02:35:11.002ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
51942025-01-15T02:35:11.002ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51952025-01-15T02:35:11.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51962025-01-15T02:35:11.005ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
51972025-01-15T02:35:11.005ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "609e4d86-2fc2-4e15-842d-958f9154d618", "content-length": "2", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
51982025-01-15T02:35:11.005ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
51992025-01-15T02:35:11.005ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fa0bebe-dc73-4d66-9e23-6472cd6abd56", "content-length": "52", "date": "Wed, 15 Jan 2025 02:35:10 GMT"} })
52002025-01-15T02:35:11.005ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
52012025-01-15T02:35:11.005ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
52022025-01-15T02:35:11.044ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "24895966-f51e-48bd-aa2b-4f1e79f915bf", "content-length": "691", "date": "Wed, 15 Jan 2025 02:35:11 GMT"} })
52032025-01-15T02:35:11.044ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
52042025-01-15T02:35:11.044ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
52052025-01-15T02:35:11.044ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
52062025-01-15T02:35:11.047ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 542.098688ms iteration = 17 status = {}
52072025-01-15T02:35:11.136ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c9ccb01c-df24-471d-be8b-e22661354498 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52082025-01-15T02:35:11.137ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c9ccb01c-df24-471d-be8b-e22661354498 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52092025-01-15T02:35:11.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2592 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c9ccb01c-df24-471d-be8b-e22661354498 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52102025-01-15T02:35:11.144ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "da38f92a-7ac7-4cc3-9890-0ea6085e17ac", "date": "Wed, 15 Jan 2025 02:35:11 GMT"} })
52112025-01-15T02:35:11.144ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
52122025-01-15T02:35:11.144ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
52132025-01-15T02:35:11.145ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "df29e2a1-7650-4280-86ef-51559b00b0a5", "content-length": "682", "date": "Wed, 15 Jan 2025 02:35:11 GMT"} })
52142025-01-15T02:35:11.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52152025-01-15T02:35:11.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52162025-01-15T02:35:11.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52172025-01-15T02:35:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52182025-01-15T02:35:11.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52192025-01-15T02:35:11.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52202025-01-15T02:35:11.155ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52212025-01-15T02:35:11.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52222025-01-15T02:35:11.162ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 657.186064ms iteration = 17 status = {}
52232025-01-15T02:35:12.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 43a60f66-ddcf-493e-8472-5dda9ad76624 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52242025-01-15T02:35:12.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 43a60f66-ddcf-493e-8472-5dda9ad76624 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52252025-01-15T02:35:12.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2048 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 43a60f66-ddcf-493e-8472-5dda9ad76624 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52262025-01-15T02:35:13.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a74b1a38-f42e-4b88-aa13-95b2caba4b86 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52272025-01-15T02:35:13.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a74b1a38-f42e-4b88-aa13-95b2caba4b86 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52282025-01-15T02:35:13.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2178 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a74b1a38-f42e-4b88-aa13-95b2caba4b86 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52292025-01-15T02:35:14.202ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 030bf603-b18d-42b9-bb12-9cf9acd47746 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52302025-01-15T02:35:14.202ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 030bf603-b18d-42b9-bb12-9cf9acd47746 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52312025-01-15T02:35:14.204ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2320 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 030bf603-b18d-42b9-bb12-9cf9acd47746 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52322025-01-15T02:35:15.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e888d08-5aa0-41e3-abf0-5ed3b3f06be6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52332025-01-15T02:35:15.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e888d08-5aa0-41e3-abf0-5ed3b3f06be6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52342025-01-15T02:35:15.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2624 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e888d08-5aa0-41e3-abf0-5ed3b3f06be6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52352025-01-15T02:35:15.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e07b2964-7608-4392-8675-d9b1ccfe4b0a roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
52362025-01-15T02:35:15.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e07b2964-7608-4392-8675-d9b1ccfe4b0a resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
52372025-01-15T02:35:15.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2840 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e07b2964-7608-4392-8675-d9b1ccfe4b0a response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
52382025-01-15T02:35:15.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 7275b024-b2ce-4e8f-bb5b-19ad5f4eb896 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
52392025-01-15T02:35:15.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 7275b024-b2ce-4e8f-bb5b-19ad5f4eb896 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
52402025-01-15T02:35:15.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2206 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 7275b024-b2ce-4e8f-bb5b-19ad5f4eb896 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
52412025-01-15T02:35:15.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 488 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 28d7a6a5-18fe-4c1b-91d5-4a1861cef858 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
52422025-01-15T02:35:16.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28db0274-3221-4e21-8496-145607c770a4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52432025-01-15T02:35:16.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28db0274-3221-4e21-8496-145607c770a4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52442025-01-15T02:35:16.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1364 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28db0274-3221-4e21-8496-145607c770a4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52452025-01-15T02:35:17.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 901ecd71-9db3-4b4a-939c-28080df97512 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52462025-01-15T02:35:17.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 901ecd71-9db3-4b4a-939c-28080df97512 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52472025-01-15T02:35:17.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2457 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 901ecd71-9db3-4b4a-939c-28080df97512 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52482025-01-15T02:35:18.292ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7cab548c-d516-4627-9840-80d25e50fe7d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52492025-01-15T02:35:18.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7cab548c-d516-4627-9840-80d25e50fe7d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52502025-01-15T02:35:18.294ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2114 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7cab548c-d516-4627-9840-80d25e50fe7d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52512025-01-15T02:35:19.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46b49522-84f4-4ec8-bd41-f3c4b2969c00 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52522025-01-15T02:35:19.314ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46b49522-84f4-4ec8-bd41-f3c4b2969c00 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52532025-01-15T02:35:19.316ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2261 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46b49522-84f4-4ec8-bd41-f3c4b2969c00 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52542025-01-15T02:35:20.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5be181b-983a-43e2-a9a3-0fa65d334fde roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52552025-01-15T02:35:20.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5be181b-983a-43e2-a9a3-0fa65d334fde resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52562025-01-15T02:35:20.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2277 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c5be181b-983a-43e2-a9a3-0fa65d334fde response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52572025-01-15T02:35:20.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
52582025-01-15T02:35:20.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52592025-01-15T02:35:20.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
52602025-01-15T02:35:20.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52612025-01-15T02:35:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52622025-01-15T02:35:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52632025-01-15T02:35:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52642025-01-15T02:35:20.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52652025-01-15T02:35:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52662025-01-15T02:35:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52672025-01-15T02:35:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52682025-01-15T02:35:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52692025-01-15T02:35:20.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.579813ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
52702025-01-15T02:35:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52712025-01-15T02:35:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52722025-01-15T02:35:20.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52732025-01-15T02:35:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
52742025-01-15T02:35:20.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52752025-01-15T02:35:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52762025-01-15T02:35:20.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
52772025-01-15T02:35:20.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.559442ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
52782025-01-15T02:35:21.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dcb29021-c547-4c7e-a8f3-bfb4a8884746 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52792025-01-15T02:35:21.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dcb29021-c547-4c7e-a8f3-bfb4a8884746 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52802025-01-15T02:35:21.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dcb29021-c547-4c7e-a8f3-bfb4a8884746 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52812025-01-15T02:35:22.379ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58bed847-2f35-4544-9ce7-d33c72a6575b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52822025-01-15T02:35:22.379ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58bed847-2f35-4544-9ce7-d33c72a6575b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52832025-01-15T02:35:22.381ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2494 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58bed847-2f35-4544-9ce7-d33c72a6575b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52842025-01-15T02:35:23.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa93da5f-0ff8-49ea-93e2-812338d4b126 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52852025-01-15T02:35:23.401ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa93da5f-0ff8-49ea-93e2-812338d4b126 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52862025-01-15T02:35:23.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2349 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa93da5f-0ff8-49ea-93e2-812338d4b126 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52872025-01-15T02:35:24.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d1c775b-8d5e-42a0-90bf-d39d01252ca0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52882025-01-15T02:35:24.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d1c775b-8d5e-42a0-90bf-d39d01252ca0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52892025-01-15T02:35:24.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1600 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d1c775b-8d5e-42a0-90bf-d39d01252ca0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52902025-01-15T02:35:25.448ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f1cece64-e633-4cd4-9411-4ce671734750 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52912025-01-15T02:35:25.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f1cece64-e633-4cd4-9411-4ce671734750 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52922025-01-15T02:35:25.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f1cece64-e633-4cd4-9411-4ce671734750 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52932025-01-15T02:35:25.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 457 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 4c363035-70f7-4665-bf1e-8ca4c8441118 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
52942025-01-15T02:35:26.470ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40cae75e-fc61-4840-b3e1-85ea7c207abb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52952025-01-15T02:35:26.470ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40cae75e-fc61-4840-b3e1-85ea7c207abb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52962025-01-15T02:35:26.472ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2273 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40cae75e-fc61-4840-b3e1-85ea7c207abb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52972025-01-15T02:35:27.492ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae149ab0-9359-45e0-851b-f14d3656ffa9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52982025-01-15T02:35:27.492ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae149ab0-9359-45e0-851b-f14d3656ffa9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
52992025-01-15T02:35:27.494ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1968 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae149ab0-9359-45e0-851b-f14d3656ffa9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53002025-01-15T02:35:28.514ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b3af7932-4679-49fe-9c9a-a18661681e29 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53012025-01-15T02:35:28.514ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b3af7932-4679-49fe-9c9a-a18661681e29 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53022025-01-15T02:35:28.516ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2240 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b3af7932-4679-49fe-9c9a-a18661681e29 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53032025-01-15T02:35:29.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f109060d-16d5-44e5-8032-02d7e1bce44a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53042025-01-15T02:35:29.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f109060d-16d5-44e5-8032-02d7e1bce44a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53052025-01-15T02:35:29.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2738 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f109060d-16d5-44e5-8032-02d7e1bce44a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53062025-01-15T02:35:30.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
53072025-01-15T02:35:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53082025-01-15T02:35:30.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
53092025-01-15T02:35:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53102025-01-15T02:35:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53112025-01-15T02:35:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53122025-01-15T02:35:30.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53132025-01-15T02:35:30.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53142025-01-15T02:35:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53152025-01-15T02:35:30.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.973187ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53162025-01-15T02:35:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53172025-01-15T02:35:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53182025-01-15T02:35:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53192025-01-15T02:35:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53202025-01-15T02:35:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53212025-01-15T02:35:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53222025-01-15T02:35:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
53232025-01-15T02:35:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53242025-01-15T02:35:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53252025-01-15T02:35:30.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
53262025-01-15T02:35:30.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.080738ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
53272025-01-15T02:35:30.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ab9969f-8c11-4dd8-9fb8-191af12c631a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53282025-01-15T02:35:30.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ab9969f-8c11-4dd8-9fb8-191af12c631a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53292025-01-15T02:35:30.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1861 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ab9969f-8c11-4dd8-9fb8-191af12c631a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53302025-01-15T02:35:30.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = d2948db6-ef6b-4bd2-937c-900e4e3502f0 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
53312025-01-15T02:35:30.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = d2948db6-ef6b-4bd2-937c-900e4e3502f0 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
53322025-01-15T02:35:30.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2587 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = d2948db6-ef6b-4bd2-937c-900e4e3502f0 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
53332025-01-15T02:35:30.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e09f8f05-7ae2-47e8-a1db-a46f5afd7ca5 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
53342025-01-15T02:35:30.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e09f8f05-7ae2-47e8-a1db-a46f5afd7ca5 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
53352025-01-15T02:35:30.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1861 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = e09f8f05-7ae2-47e8-a1db-a46f5afd7ca5 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
53362025-01-15T02:35:31.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d8fef3aa-c190-4dfc-a07c-2586fd9af819 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53372025-01-15T02:35:31.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d8fef3aa-c190-4dfc-a07c-2586fd9af819 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53382025-01-15T02:35:31.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2143 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d8fef3aa-c190-4dfc-a07c-2586fd9af819 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53392025-01-15T02:35:32.603ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3be7c552-41c2-4e97-8339-a40788c82935 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53402025-01-15T02:35:32.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3be7c552-41c2-4e97-8339-a40788c82935 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53412025-01-15T02:35:32.605ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1923 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3be7c552-41c2-4e97-8339-a40788c82935 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53422025-01-15T02:35:33.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 66bbb9ac-7356-4805-814a-f4bc99fbc39d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53432025-01-15T02:35:33.626ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 66bbb9ac-7356-4805-814a-f4bc99fbc39d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53442025-01-15T02:35:33.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2152 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 66bbb9ac-7356-4805-814a-f4bc99fbc39d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53452025-01-15T02:35:34.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 465d8e7c-d449-4cb5-ba3d-0328e99b805a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53462025-01-15T02:35:34.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 465d8e7c-d449-4cb5-ba3d-0328e99b805a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53472025-01-15T02:35:34.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 465d8e7c-d449-4cb5-ba3d-0328e99b805a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53482025-01-15T02:35:35.669ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfeb4115-1ce7-436e-a14c-985b418fdbe7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53492025-01-15T02:35:35.669ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfeb4115-1ce7-436e-a14c-985b418fdbe7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53502025-01-15T02:35:35.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1610 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfeb4115-1ce7-436e-a14c-985b418fdbe7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53512025-01-15T02:35:35.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 450 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 40714cca-ba26-45e2-abdc-410863290788 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
53522025-01-15T02:35:36.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c2ae1170-ef17-4bf6-a8fc-b851366fb1b5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53532025-01-15T02:35:36.691ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c2ae1170-ef17-4bf6-a8fc-b851366fb1b5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53542025-01-15T02:35:36.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c2ae1170-ef17-4bf6-a8fc-b851366fb1b5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53552025-01-15T02:35:37.712ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 175cd287-9126-4cb1-b290-d3cff97d09f5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53562025-01-15T02:35:37.712ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 175cd287-9126-4cb1-b290-d3cff97d09f5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53572025-01-15T02:35:37.714ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1999 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 175cd287-9126-4cb1-b290-d3cff97d09f5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53582025-01-15T02:35:38.734ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 041f47a3-2c7a-4d95-94c4-7bbc499f5681 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53592025-01-15T02:35:38.734ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 041f47a3-2c7a-4d95-94c4-7bbc499f5681 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53602025-01-15T02:35:38.736ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 041f47a3-2c7a-4d95-94c4-7bbc499f5681 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53612025-01-15T02:35:39.756ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46bd852b-8dac-414f-a1d3-7009c46a0b60 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53622025-01-15T02:35:39.756ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46bd852b-8dac-414f-a1d3-7009c46a0b60 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53632025-01-15T02:35:39.758ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2527 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46bd852b-8dac-414f-a1d3-7009c46a0b60 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
53642025-01-15T02:35:40.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
53652025-01-15T02:35:40.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
53662025-01-15T02:35:40.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
53672025-01-15T02:35:40.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
53682025-01-15T02:35:40.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
53692025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
53702025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
53712025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
53722025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
53732025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
53742025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53752025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53762025-01-15T02:35:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53772025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
53782025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
53792025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53802025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53812025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
53822025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
53832025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53842025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
53852025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53862025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
53872025-01-15T02:35:40.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
53882025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53892025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
53902025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53912025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
53922025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
53932025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53942025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53952025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53962025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
53972025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53982025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53992025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
54002025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54012025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54022025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54032025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
54042025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54052025-01-15T02:35:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
54062025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
54072025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
54082025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54092025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
54102025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
54112025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
54122025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
54132025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
54142025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
54152025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
54162025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
54172025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54182025-01-15T02:35:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3827, tv_nsec: 343664008 } })
54192025-01-15T02:35:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3827, tv_nsec: 343695279 } })
54202025-01-15T02:35:40.506ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.127685ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
54212025-01-15T02:35:40.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
54222025-01-15T02:35:40.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
54232025-01-15T02:35:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54242025-01-15T02:35:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54252025-01-15T02:35:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54262025-01-15T02:35:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54272025-01-15T02:35:40.508ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54282025-01-15T02:35:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
54292025-01-15T02:35:40.509ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
54302025-01-15T02:35:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
54312025-01-15T02:35:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
54322025-01-15T02:35:40.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54332025-01-15T02:35:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54342025-01-15T02:35:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54352025-01-15T02:35:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54362025-01-15T02:35:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54372025-01-15T02:35:40.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54382025-01-15T02:35:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54392025-01-15T02:35:40.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54402025-01-15T02:35:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54412025-01-15T02:35:40.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54422025-01-15T02:35:40.520ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3827, tv_nsec: 356988498 } })
54432025-01-15T02:35:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54442025-01-15T02:35:40.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54452025-01-15T02:35:40.520ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
54462025-01-15T02:35:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54472025-01-15T02:35:40.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54482025-01-15T02:35:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54492025-01-15T02:35:40.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54502025-01-15T02:35:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54512025-01-15T02:35:40.558ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
54522025-01-15T02:35:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
54532025-01-15T02:35:40.560ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54542025-01-15T02:35:40.560ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54552025-01-15T02:35:40.561ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54562025-01-15T02:35:40.561ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
54572025-01-15T02:35:40.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54582025-01-15T02:35:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
54592025-01-15T02:35:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
54602025-01-15T02:35:40.562ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
54612025-01-15T02:35:40.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
54622025-01-15T02:35:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
54632025-01-15T02:35:40.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54642025-01-15T02:35:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54652025-01-15T02:35:40.565ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.580168ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
54662025-01-15T02:35:40.567ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54672025-01-15T02:35:40.567ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54682025-01-15T02:35:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54692025-01-15T02:35:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54702025-01-15T02:35:40.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54712025-01-15T02:35:40.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 62.421374ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
54722025-01-15T02:35:40.568ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
54732025-01-15T02:35:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
54742025-01-15T02:35:40.569ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54752025-01-15T02:35:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54762025-01-15T02:35:40.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54772025-01-15T02:35:40.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54782025-01-15T02:35:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54792025-01-15T02:35:40.571ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.238157ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54802025-01-15T02:35:40.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54812025-01-15T02:35:40.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 65.97492ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54822025-01-15T02:35:40.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54832025-01-15T02:35:40.575ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54842025-01-15T02:35:40.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54852025-01-15T02:35:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54862025-01-15T02:35:40.576ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
54872025-01-15T02:35:40.576ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 71.403326ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
54882025-01-15T02:35:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
54892025-01-15T02:35:40.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54902025-01-15T02:35:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54912025-01-15T02:35:40.589ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
54922025-01-15T02:35:40.589ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.348006ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
54932025-01-15T02:35:40.590ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54942025-01-15T02:35:40.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54952025-01-15T02:35:40.592ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54962025-01-15T02:35:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54972025-01-15T02:35:40.594ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 88.927169ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54982025-01-15T02:35:40.594ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54992025-01-15T02:35:40.596ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55002025-01-15T02:35:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55012025-01-15T02:35:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
55022025-01-15T02:35:40.597ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55032025-01-15T02:35:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55042025-01-15T02:35:40.598ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55052025-01-15T02:35:40.599ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 94.227083ms iteration = 18 status = 0
55062025-01-15T02:35:40.600ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
55072025-01-15T02:35:40.600ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55082025-01-15T02:35:40.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55092025-01-15T02:35:40.605ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
55102025-01-15T02:35:40.605ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55112025-01-15T02:35:40.605ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38bbfe3f-f95d-4ad0-9793-e7cd1e608e8e", "content-length": "26", "date": "Wed, 15 Jan 2025 02:35:39 GMT"} })
55122025-01-15T02:35:40.605ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55132025-01-15T02:35:40.606ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55142025-01-15T02:35:40.606ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
55152025-01-15T02:35:40.606ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55162025-01-15T02:35:40.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55172025-01-15T02:35:40.608ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55182025-01-15T02:35:40.611ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
55192025-01-15T02:35:40.611ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55202025-01-15T02:35:40.612ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa86189e-958f-4552-927d-888f3a853761", "content-length": "26", "date": "Wed, 15 Jan 2025 02:35:39 GMT"} })
55212025-01-15T02:35:40.612ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55222025-01-15T02:35:40.613ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
55232025-01-15T02:35:40.613ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55242025-01-15T02:35:40.613ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "079b7171-dc60-4555-8079-3657e0cd90d2", "content-length": "26", "date": "Wed, 15 Jan 2025 02:35:39 GMT"} })
55252025-01-15T02:35:40.613ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55262025-01-15T02:35:40.614ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
55272025-01-15T02:35:40.615ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55282025-01-15T02:35:40.615ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 109.146665ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
55292025-01-15T02:35:40.640ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55302025-01-15T02:35:40.640ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
55312025-01-15T02:35:40.640ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 135.049505ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
55322025-01-15T02:35:40.644ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55332025-01-15T02:35:40.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55342025-01-15T02:35:40.645ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
55352025-01-15T02:35:40.645ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 140.658156ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
55362025-01-15T02:35:40.664ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 159.389021ms iteration = 20 status = {"physical_disks_added":0}
55372025-01-15T02:35:40.686ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55382025-01-15T02:35:40.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55392025-01-15T02:35:40.692ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 188.868846ms iteration = 18 status = {}
55402025-01-15T02:35:40.706ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
55412025-01-15T02:35:40.707ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "021a3cf1-cad6-4fcb-b666-cf5f8f82f290", "content-length": "1", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55422025-01-15T02:35:40.708ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
55432025-01-15T02:35:40.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
55442025-01-15T02:35:40.716ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
55452025-01-15T02:35:40.716ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
55462025-01-15T02:35:40.716ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c01174b-0f0a-468c-b48b-7310029f34d9", "content-length": "372", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55472025-01-15T02:35:40.716ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55482025-01-15T02:35:40.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55492025-01-15T02:35:40.717ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55502025-01-15T02:35:40.718ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 214.271582ms iteration = 18 status = null
55512025-01-15T02:35:40.719ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
55522025-01-15T02:35:40.719ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55532025-01-15T02:35:40.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55542025-01-15T02:35:40.720ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55552025-01-15T02:35:40.722ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
55562025-01-15T02:35:40.722ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55572025-01-15T02:35:40.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55582025-01-15T02:35:40.761ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
55592025-01-15T02:35:40.761ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
55602025-01-15T02:35:40.762ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1ccead9f-9cf9-4feb-914e-b738233843e8", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55612025-01-15T02:35:40.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55622025-01-15T02:35:40.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55632025-01-15T02:35:40.764ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55642025-01-15T02:35:40.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55652025-01-15T02:35:40.768ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55662025-01-15T02:35:40.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55672025-01-15T02:35:40.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55682025-01-15T02:35:40.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55692025-01-15T02:35:40.771ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
55702025-01-15T02:35:40.771ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
55712025-01-15T02:35:40.771ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
55722025-01-15T02:35:40.771ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55732025-01-15T02:35:40.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55742025-01-15T02:35:40.773ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "65e9ebe5-f234-4e44-95bd-ecfd43b0d66f", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55752025-01-15T02:35:40.774ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 269.596805ms iteration = 18 status = {}
55762025-01-15T02:35:40.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6efa4add-5398-44c8-9072-e58ffdf5168c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
55772025-01-15T02:35:40.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6efa4add-5398-44c8-9072-e58ffdf5168c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
55782025-01-15T02:35:40.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6efa4add-5398-44c8-9072-e58ffdf5168c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
55792025-01-15T02:35:40.781ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
55802025-01-15T02:35:40.781ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "49e22cf9-7fcd-40bc-a856-bb3aa4e2e5c4", "content-length": "134", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55812025-01-15T02:35:40.781ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
55822025-01-15T02:35:40.781ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
55832025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
55842025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
55852025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55862025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55872025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55882025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
55892025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55902025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
55912025-01-15T02:35:40.782ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
55922025-01-15T02:35:40.782ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55932025-01-15T02:35:40.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55942025-01-15T02:35:40.785ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
55952025-01-15T02:35:40.789ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "26640ec9-e34d-4937-aa3e-4b4015727441", "content-length": "2", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55962025-01-15T02:35:40.790ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
55972025-01-15T02:35:40.790ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4aaf4042-cf20-4bbb-be30-7bfaf085132d", "content-length": "52", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
55982025-01-15T02:35:40.790ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
55992025-01-15T02:35:40.790ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
56002025-01-15T02:35:40.819ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
56012025-01-15T02:35:40.829ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b233d8c3-e5be-43fb-a16d-5db8b8a6ef53", "content-length": "691", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
56022025-01-15T02:35:40.829ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
56032025-01-15T02:35:40.829ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
56042025-01-15T02:35:40.829ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
56052025-01-15T02:35:40.831ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 326.911322ms iteration = 18 status = {}
56062025-01-15T02:35:40.883ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e2be86ed-9f00-4cd8-8cc0-a6b0bf7d4797", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
56072025-01-15T02:35:40.883ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
56082025-01-15T02:35:40.883ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
56092025-01-15T02:35:40.886ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe2949c-26f7-440f-8f87-34c04258330a", "content-length": "682", "date": "Wed, 15 Jan 2025 02:35:40 GMT"} })
56102025-01-15T02:35:40.887ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56112025-01-15T02:35:40.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56122025-01-15T02:35:40.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56132025-01-15T02:35:40.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56142025-01-15T02:35:40.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56152025-01-15T02:35:40.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56162025-01-15T02:35:40.896ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56172025-01-15T02:35:40.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56182025-01-15T02:35:40.907ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 401.985517ms iteration = 18 status = {}
56192025-01-15T02:35:41.085ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
56202025-01-15T02:35:41.085ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
56212025-01-15T02:35:41.116ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
56222025-01-15T02:35:41.116ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:65220
56232025-01-15T02:35:41.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:65220 req_id = 938544f9-ae3b-4ad9-a436-3bc5937a8980 roles = RoleSet { roles: {} } uri = /metrics/producers
56242025-01-15T02:35:41.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:65220 req_id = 938544f9-ae3b-4ad9-a436-3bc5937a8980 resource = Database result = Ok(()) uri = /metrics/producers
56252025-01-15T02:35:41.174ZINFOnexus: assigned collector to new producer collector_id = f71d33aa-7320-48b9-81d2-899a100e78cd file = nexus/src/app/oximeter.rs:76 producer_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
56262025-01-15T02:35:41.174ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57942 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:65220 req_id = 938544f9-ae3b-4ad9-a436-3bc5937a8980 response_code = 201 uri = /metrics/producers
56272025-01-15T02:35:41.175ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "938544f9-ae3b-4ad9-a436-3bc5937a8980", "content-length": "41", "date": "Wed, 15 Jan 2025 02:35:41 GMT"} })
56282025-01-15T02:35:41.176ZDEBGnexus (producer-registration-task): registered with nexus successfully
56292025-01-15T02:35:41.176ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
56302025-01-15T02:35:41.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5a5da0b9-cdab-4682-9ff2-c11f46e05ad3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56312025-01-15T02:35:41.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5a5da0b9-cdab-4682-9ff2-c11f46e05ad3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56322025-01-15T02:35:41.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2035 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5a5da0b9-cdab-4682-9ff2-c11f46e05ad3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56332025-01-15T02:35:42.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d205e483-cc13-4d8a-ab33-988e7bbbf080 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56342025-01-15T02:35:42.824ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d205e483-cc13-4d8a-ab33-988e7bbbf080 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56352025-01-15T02:35:42.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2113 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d205e483-cc13-4d8a-ab33-988e7bbbf080 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56362025-01-15T02:35:43.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0aaedc3e-56f5-499f-bbce-12b725ec0378 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56372025-01-15T02:35:43.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0aaedc3e-56f5-499f-bbce-12b725ec0378 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56382025-01-15T02:35:43.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2434 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0aaedc3e-56f5-499f-bbce-12b725ec0378 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56392025-01-15T02:35:44.867ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 638343b8-faec-4050-b528-db3158cc5e87 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56402025-01-15T02:35:44.867ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 638343b8-faec-4050-b528-db3158cc5e87 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56412025-01-15T02:35:44.869ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2491 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 638343b8-faec-4050-b528-db3158cc5e87 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56422025-01-15T02:35:45.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 72de0196-f859-4704-b9c3-01bf974ff0f2 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
56432025-01-15T02:35:45.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 72de0196-f859-4704-b9c3-01bf974ff0f2 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
56442025-01-15T02:35:45.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2217 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 72de0196-f859-4704-b9c3-01bf974ff0f2 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
56452025-01-15T02:35:45.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = b2759d03-e85b-4005-97ec-f0063eb2c5fe roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
56462025-01-15T02:35:45.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = b2759d03-e85b-4005-97ec-f0063eb2c5fe resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
56472025-01-15T02:35:45.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2329 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = b2759d03-e85b-4005-97ec-f0063eb2c5fe response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
56482025-01-15T02:35:45.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 488 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = b92d9f9c-2259-48f9-886c-bd45b0a46fb1 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
56492025-01-15T02:35:45.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89cf29d5-f682-4f9f-a608-4ad26f8f4c4c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56502025-01-15T02:35:45.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89cf29d5-f682-4f9f-a608-4ad26f8f4c4c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56512025-01-15T02:35:45.891ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2178 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89cf29d5-f682-4f9f-a608-4ad26f8f4c4c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56522025-01-15T02:35:46.910ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f61df457-626b-4e84-9860-5bd98674fd8f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56532025-01-15T02:35:46.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f61df457-626b-4e84-9860-5bd98674fd8f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56542025-01-15T02:35:46.913ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2627 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f61df457-626b-4e84-9860-5bd98674fd8f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56552025-01-15T02:35:47.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8e401a8c-1dfd-4ac1-99ac-d06398195c4b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56562025-01-15T02:35:47.934ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8e401a8c-1dfd-4ac1-99ac-d06398195c4b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56572025-01-15T02:35:47.936ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2119 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8e401a8c-1dfd-4ac1-99ac-d06398195c4b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56582025-01-15T02:35:48.956ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b485bcee-8eac-470b-a4d8-7324f3dd30c9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56592025-01-15T02:35:48.956ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b485bcee-8eac-470b-a4d8-7324f3dd30c9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56602025-01-15T02:35:48.957ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1723 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b485bcee-8eac-470b-a4d8-7324f3dd30c9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56612025-01-15T02:35:49.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6e10c543-c9c2-42a7-ae08-789f860a8ff4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56622025-01-15T02:35:49.978ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6e10c543-c9c2-42a7-ae08-789f860a8ff4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56632025-01-15T02:35:49.979ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1922 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6e10c543-c9c2-42a7-ae08-789f860a8ff4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56642025-01-15T02:35:50.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
56652025-01-15T02:35:50.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
56662025-01-15T02:35:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56672025-01-15T02:35:50.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56682025-01-15T02:35:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56692025-01-15T02:35:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56702025-01-15T02:35:50.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56712025-01-15T02:35:50.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56722025-01-15T02:35:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56732025-01-15T02:35:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56742025-01-15T02:35:50.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56752025-01-15T02:35:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56762025-01-15T02:35:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56772025-01-15T02:35:50.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56782025-01-15T02:35:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56792025-01-15T02:35:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
56802025-01-15T02:35:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56812025-01-15T02:35:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56822025-01-15T02:35:50.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
56832025-01-15T02:35:50.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.399277ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
56842025-01-15T02:35:50.535ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.829152ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56852025-01-15T02:35:50.999ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40cceef5-302c-47b8-bb14-2525a1899be2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56862025-01-15T02:35:51.000ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40cceef5-302c-47b8-bb14-2525a1899be2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56872025-01-15T02:35:51.001ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40cceef5-302c-47b8-bb14-2525a1899be2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56882025-01-15T02:35:52.021ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ddf2fb2f-97e2-4fec-951d-9de196318ae5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56892025-01-15T02:35:52.021ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ddf2fb2f-97e2-4fec-951d-9de196318ae5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56902025-01-15T02:35:52.023ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ddf2fb2f-97e2-4fec-951d-9de196318ae5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56912025-01-15T02:35:53.043ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e54e71f-83ac-4299-8757-0b36bd844ff3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56922025-01-15T02:35:53.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e54e71f-83ac-4299-8757-0b36bd844ff3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56932025-01-15T02:35:53.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1976 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2e54e71f-83ac-4299-8757-0b36bd844ff3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56942025-01-15T02:35:54.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1fd9170f-57c1-4e6c-8a25-3cacdd18f408 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56952025-01-15T02:35:54.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1fd9170f-57c1-4e6c-8a25-3cacdd18f408 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56962025-01-15T02:35:54.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1fd9170f-57c1-4e6c-8a25-3cacdd18f408 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56972025-01-15T02:35:55.086ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7e2d876-d9f4-4490-8d97-9e8d07bdc006 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56982025-01-15T02:35:55.087ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7e2d876-d9f4-4490-8d97-9e8d07bdc006 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
56992025-01-15T02:35:55.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2350 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7e2d876-d9f4-4490-8d97-9e8d07bdc006 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57002025-01-15T02:35:55.747ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 486 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = cad41b9c-56ef-475f-b1e2-a327a3f2f9db response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
57012025-01-15T02:35:56.109ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = edc9ec00-2fd0-46cc-9a7c-413dc2b445f7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57022025-01-15T02:35:56.109ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = edc9ec00-2fd0-46cc-9a7c-413dc2b445f7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57032025-01-15T02:35:56.111ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2005 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = edc9ec00-2fd0-46cc-9a7c-413dc2b445f7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57042025-01-15T02:35:57.131ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32023f8f-9ca5-4122-81b0-edc9563b531e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57052025-01-15T02:35:57.131ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32023f8f-9ca5-4122-81b0-edc9563b531e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57062025-01-15T02:35:57.133ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2255 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32023f8f-9ca5-4122-81b0-edc9563b531e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57072025-01-15T02:35:58.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f0caf41f-8c9c-4f4c-81f8-8f43f113761d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57082025-01-15T02:35:58.155ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f0caf41f-8c9c-4f4c-81f8-8f43f113761d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57092025-01-15T02:35:58.156ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1579 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f0caf41f-8c9c-4f4c-81f8-8f43f113761d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57102025-01-15T02:35:59.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4d304316-e07e-4a1f-88d5-c03063c1caa7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57112025-01-15T02:35:59.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4d304316-e07e-4a1f-88d5-c03063c1caa7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57122025-01-15T02:35:59.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2453 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4d304316-e07e-4a1f-88d5-c03063c1caa7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57132025-01-15T02:36:00.199ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e5006755-fc9d-4b56-8b66-f1828d9a616d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57142025-01-15T02:36:00.200ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e5006755-fc9d-4b56-8b66-f1828d9a616d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57152025-01-15T02:36:00.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2198 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e5006755-fc9d-4b56-8b66-f1828d9a616d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57162025-01-15T02:36:00.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
57172025-01-15T02:36:00.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
57182025-01-15T02:36:00.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57192025-01-15T02:36:00.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57202025-01-15T02:36:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57212025-01-15T02:36:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57222025-01-15T02:36:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57232025-01-15T02:36:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57242025-01-15T02:36:00.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57252025-01-15T02:36:00.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.043889ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
57262025-01-15T02:36:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57272025-01-15T02:36:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57282025-01-15T02:36:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57292025-01-15T02:36:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57302025-01-15T02:36:00.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57312025-01-15T02:36:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57322025-01-15T02:36:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
57332025-01-15T02:36:00.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57342025-01-15T02:36:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57352025-01-15T02:36:00.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
57362025-01-15T02:36:00.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.447129ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
57372025-01-15T02:36:01.222ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7aecec8-10e7-4c9d-8ef5-b80832b9c2d7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57382025-01-15T02:36:01.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7aecec8-10e7-4c9d-8ef5-b80832b9c2d7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57392025-01-15T02:36:01.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2619 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7aecec8-10e7-4c9d-8ef5-b80832b9c2d7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57402025-01-15T02:36:02.245ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fa06a61f-eaba-4211-89ab-d81262991e1c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57412025-01-15T02:36:02.245ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fa06a61f-eaba-4211-89ab-d81262991e1c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57422025-01-15T02:36:02.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1777 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fa06a61f-eaba-4211-89ab-d81262991e1c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57432025-01-15T02:36:03.266ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4c374d24-ab28-4b85-8ecf-740a4101fa13 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57442025-01-15T02:36:03.267ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4c374d24-ab28-4b85-8ecf-740a4101fa13 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57452025-01-15T02:36:03.269ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2108 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4c374d24-ab28-4b85-8ecf-740a4101fa13 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57462025-01-15T02:36:04.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a2081510-6cac-4a4a-88c0-517a23b5c99b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57472025-01-15T02:36:04.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a2081510-6cac-4a4a-88c0-517a23b5c99b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57482025-01-15T02:36:04.289ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2244 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a2081510-6cac-4a4a-88c0-517a23b5c99b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57492025-01-15T02:36:05.308ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58366c16-b031-4c7b-b622-70b8d7243ae3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57502025-01-15T02:36:05.308ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58366c16-b031-4c7b-b622-70b8d7243ae3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57512025-01-15T02:36:05.310ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2147 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58366c16-b031-4c7b-b622-70b8d7243ae3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57522025-01-15T02:36:05.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 487 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 31c6d99b-2fe9-46c7-b2b5-024721c50583 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
57532025-01-15T02:36:06.329ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4d4e71c-1abc-4afc-8e1e-71713fbaa934 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57542025-01-15T02:36:06.330ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4d4e71c-1abc-4afc-8e1e-71713fbaa934 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57552025-01-15T02:36:06.332ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2350 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4d4e71c-1abc-4afc-8e1e-71713fbaa934 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57562025-01-15T02:36:07.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 641b37b3-6c09-469b-a390-418bdcc0e005 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57572025-01-15T02:36:07.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 641b37b3-6c09-469b-a390-418bdcc0e005 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57582025-01-15T02:36:07.354ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2089 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 641b37b3-6c09-469b-a390-418bdcc0e005 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57592025-01-15T02:36:08.374ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf424f66-ec34-42ea-8376-26ed3695f34f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57602025-01-15T02:36:08.374ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf424f66-ec34-42ea-8376-26ed3695f34f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57612025-01-15T02:36:08.376ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2111 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cf424f66-ec34-42ea-8376-26ed3695f34f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57622025-01-15T02:36:09.395ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 60bd62a0-5641-4e97-8019-e9291b31079d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57632025-01-15T02:36:09.396ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 60bd62a0-5641-4e97-8019-e9291b31079d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57642025-01-15T02:36:09.397ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2081 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 60bd62a0-5641-4e97-8019-e9291b31079d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57652025-01-15T02:36:10.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ebe69ba0-e8c8-419b-93f0-cc428a0af8ca roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57662025-01-15T02:36:10.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ebe69ba0-e8c8-419b-93f0-cc428a0af8ca resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57672025-01-15T02:36:10.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2612 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ebe69ba0-e8c8-419b-93f0-cc428a0af8ca response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
57682025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
57692025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
57702025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
57712025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
57722025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
57732025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
57742025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
57752025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
57762025-01-15T02:36:10.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
57772025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
57782025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
57792025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
57802025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
57812025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
57822025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
57832025-01-15T02:36:10.504ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:26:10.503886097 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
57842025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
57852025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
57862025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
57872025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
57882025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
57892025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
57902025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
57912025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
57922025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57932025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
57942025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57952025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
57962025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
57972025-01-15T02:36:10.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
57982025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57992025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
58002025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
58012025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
58022025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
58032025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58042025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
58052025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58062025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58072025-01-15T02:36:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3857, tv_nsec: 341557001 } })
58082025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
58092025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
58102025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58112025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
58122025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
58132025-01-15T02:36:10.504ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
58142025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
58152025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
58162025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.49\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
58172025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58182025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
58192025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
58202025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
58212025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58222025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
58232025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58242025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
58252025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
58262025-01-15T02:36:10.504ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
58272025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58282025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
58292025-01-15T02:36:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3857, tv_nsec: 341820619 } })
58302025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58312025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58322025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
58332025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58342025-01-15T02:36:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58352025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58362025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58372025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
58382025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58392025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
58402025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.247203ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
58412025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
58422025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
58432025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58442025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
58452025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58462025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
58472025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58482025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58492025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58502025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58512025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
58522025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.690056ms iteration = 10 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
58532025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
58542025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
58552025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58562025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
58572025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
58582025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
58592025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
58602025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
58612025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
58622025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
58632025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
58642025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58652025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58662025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58672025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58682025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
58692025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
58702025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
58712025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
58722025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
58732025-01-15T02:36:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58742025-01-15T02:36:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
58752025-01-15T02:36:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
58762025-01-15T02:36:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
58772025-01-15T02:36:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
58782025-01-15T02:36:10.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58792025-01-15T02:36:10.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3857, tv_nsec: 344878881 } })
58802025-01-15T02:36:10.508ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
58812025-01-15T02:36:10.509ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
58822025-01-15T02:36:10.509ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
58832025-01-15T02:36:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
58842025-01-15T02:36:10.509ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.826058ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
58852025-01-15T02:36:10.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3857, tv_nsec: 346762402 } })
58862025-01-15T02:36:10.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
58872025-01-15T02:36:10.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58882025-01-15T02:36:10.510ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:26:10.503886097 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
58892025-01-15T02:36:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.318431ms iteration = 10 status = {"expiration":"2025-01-15T02:26:10.503886097Z","pruned":[]}
58902025-01-15T02:36:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58912025-01-15T02:36:10.512ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58922025-01-15T02:36:10.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58932025-01-15T02:36:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58942025-01-15T02:36:10.513ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58952025-01-15T02:36:10.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58962025-01-15T02:36:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
58972025-01-15T02:36:10.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58982025-01-15T02:36:10.514ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58992025-01-15T02:36:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59002025-01-15T02:36:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
59012025-01-15T02:36:10.517ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59022025-01-15T02:36:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
59032025-01-15T02:36:10.517ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
59042025-01-15T02:36:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59052025-01-15T02:36:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
59062025-01-15T02:36:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
59072025-01-15T02:36:10.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59082025-01-15T02:36:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59092025-01-15T02:36:10.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59102025-01-15T02:36:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59112025-01-15T02:36:10.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59122025-01-15T02:36:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59132025-01-15T02:36:10.527ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59142025-01-15T02:36:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
59152025-01-15T02:36:10.530ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
59162025-01-15T02:36:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
59172025-01-15T02:36:10.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59182025-01-15T02:36:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
59192025-01-15T02:36:10.537ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
59202025-01-15T02:36:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
59212025-01-15T02:36:10.540ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 35.317323ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
59222025-01-15T02:36:10.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59232025-01-15T02:36:10.542ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
59242025-01-15T02:36:10.542ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
59252025-01-15T02:36:10.542ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 37.4462ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
59262025-01-15T02:36:10.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59272025-01-15T02:36:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59282025-01-15T02:36:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
59292025-01-15T02:36:10.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59302025-01-15T02:36:10.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59312025-01-15T02:36:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59322025-01-15T02:36:10.548ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59332025-01-15T02:36:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59342025-01-15T02:36:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59352025-01-15T02:36:10.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59362025-01-15T02:36:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59372025-01-15T02:36:10.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59382025-01-15T02:36:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
59392025-01-15T02:36:10.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59402025-01-15T02:36:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59412025-01-15T02:36:10.566ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
59422025-01-15T02:36:10.566ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 61.847449ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
59432025-01-15T02:36:10.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59442025-01-15T02:36:10.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59452025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
59462025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59472025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59482025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59492025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59502025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59512025-01-15T02:36:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59522025-01-15T02:36:10.573ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59532025-01-15T02:36:10.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59542025-01-15T02:36:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59552025-01-15T02:36:10.578ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59562025-01-15T02:36:10.578ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59572025-01-15T02:36:10.582ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59582025-01-15T02:36:10.582ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
59592025-01-15T02:36:10.582ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59602025-01-15T02:36:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
59612025-01-15T02:36:10.582ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
59622025-01-15T02:36:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
59632025-01-15T02:36:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59642025-01-15T02:36:10.583ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3857, tv_nsec: 420209304 } })
59652025-01-15T02:36:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
59662025-01-15T02:36:10.583ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59672025-01-15T02:36:10.583ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
59682025-01-15T02:36:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59692025-01-15T02:36:10.584ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59702025-01-15T02:36:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59712025-01-15T02:36:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 80.950905ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
59722025-01-15T02:36:10.588ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59732025-01-15T02:36:10.590ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59742025-01-15T02:36:10.590ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59752025-01-15T02:36:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
59762025-01-15T02:36:10.590ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
59772025-01-15T02:36:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
59782025-01-15T02:36:10.592ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59792025-01-15T02:36:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59802025-01-15T02:36:10.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59812025-01-15T02:36:10.593ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59822025-01-15T02:36:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59832025-01-15T02:36:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
59842025-01-15T02:36:10.594ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
59852025-01-15T02:36:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
59862025-01-15T02:36:10.598ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59872025-01-15T02:36:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59882025-01-15T02:36:10.602ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59892025-01-15T02:36:10.604ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 99.610448ms iteration = 5 status = {"nsuccess":5}
59902025-01-15T02:36:10.604ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59912025-01-15T02:36:10.604ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
59922025-01-15T02:36:10.604ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 100.092901ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
59932025-01-15T02:36:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59942025-01-15T02:36:10.605ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 99.766312ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
59952025-01-15T02:36:10.606ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
59962025-01-15T02:36:10.606ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
59972025-01-15T02:36:10.606ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
59982025-01-15T02:36:10.606ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59992025-01-15T02:36:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60002025-01-15T02:36:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
60012025-01-15T02:36:10.609ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 104.222723ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
60022025-01-15T02:36:10.615ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60032025-01-15T02:36:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60042025-01-15T02:36:10.618ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60052025-01-15T02:36:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60062025-01-15T02:36:10.623ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 118.637662ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60072025-01-15T02:36:10.628ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
60082025-01-15T02:36:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
60092025-01-15T02:36:10.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60102025-01-15T02:36:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60112025-01-15T02:36:10.640ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60122025-01-15T02:36:10.640ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
60132025-01-15T02:36:10.640ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
60142025-01-15T02:36:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
60152025-01-15T02:36:10.641ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
60162025-01-15T02:36:10.641ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
60172025-01-15T02:36:10.641ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 137.005697ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
60182025-01-15T02:36:10.643ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 138.344753ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
60192025-01-15T02:36:10.645ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60202025-01-15T02:36:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60212025-01-15T02:36:10.651ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
60222025-01-15T02:36:10.651ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
60232025-01-15T02:36:10.651ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
60242025-01-15T02:36:10.651ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
60252025-01-15T02:36:10.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
60262025-01-15T02:36:10.699ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 194.893889ms iteration = 21 status = {"physical_disks_added":0}
60272025-01-15T02:36:10.713ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
60282025-01-15T02:36:10.759ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
60292025-01-15T02:36:10.759ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60302025-01-15T02:36:10.759ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "257a0298-f77d-4ecd-983d-a30a42081738", "content-length": "26", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
60312025-01-15T02:36:10.759ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60322025-01-15T02:36:10.761ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
60332025-01-15T02:36:10.869ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
60342025-01-15T02:36:10.869ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60352025-01-15T02:36:10.870ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d220f455-b5ac-4ba1-ab38-aa058293d78e", "content-length": "26", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
60362025-01-15T02:36:10.870ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60372025-01-15T02:36:10.871ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60382025-01-15T02:36:10.888ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
60392025-01-15T02:36:10.905ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
60402025-01-15T02:36:10.905ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60412025-01-15T02:36:10.905ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbfbffe4-9792-4c7b-a2e9-6cd1386343e5", "content-length": "26", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
60422025-01-15T02:36:10.905ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60432025-01-15T02:36:10.907ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60442025-01-15T02:36:10.913ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
60452025-01-15T02:36:10.913ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60462025-01-15T02:36:10.915ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60472025-01-15T02:36:10.915ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60482025-01-15T02:36:10.916ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60492025-01-15T02:36:10.917ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60502025-01-15T02:36:10.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
60512025-01-15T02:36:10.917ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60522025-01-15T02:36:10.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60532025-01-15T02:36:10.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
60542025-01-15T02:36:10.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
60552025-01-15T02:36:10.919ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60562025-01-15T02:36:10.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60572025-01-15T02:36:10.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60582025-01-15T02:36:10.919ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60592025-01-15T02:36:10.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60602025-01-15T02:36:10.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 417.018663ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
60612025-01-15T02:36:10.953ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60622025-01-15T02:36:10.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
60632025-01-15T02:36:10.955ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60642025-01-15T02:36:10.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60652025-01-15T02:36:10.957ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60662025-01-15T02:36:10.958ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 454.028182ms iteration = 19 status = 0
60672025-01-15T02:36:10.958ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60682025-01-15T02:36:10.958ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
60692025-01-15T02:36:10.958ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 454.279359ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
60702025-01-15T02:36:10.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
60712025-01-15T02:36:10.961ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
60722025-01-15T02:36:10.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
60732025-01-15T02:36:10.964ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
60742025-01-15T02:36:10.964ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 461.145744ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
60752025-01-15T02:36:10.977ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
60762025-01-15T02:36:10.977ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
60772025-01-15T02:36:10.977ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
60782025-01-15T02:36:10.977ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 473.262731ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
60792025-01-15T02:36:10.977ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
60802025-01-15T02:36:10.977ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60812025-01-15T02:36:10.977ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a067a0e1-ff4f-45a4-b2bf-9b391fe70df0", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
60822025-01-15T02:36:10.977ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
60832025-01-15T02:36:10.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60842025-01-15T02:36:10.979ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2d7ab46d-7c11-4e84-8f1e-c63bb7c902e5", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
60852025-01-15T02:36:10.979ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
60862025-01-15T02:36:10.979ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60872025-01-15T02:36:10.982ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
60882025-01-15T02:36:10.982ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 478.374469ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
60892025-01-15T02:36:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
60902025-01-15T02:36:10.982ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60912025-01-15T02:36:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60922025-01-15T02:36:10.985ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
60932025-01-15T02:36:10.985ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
60942025-01-15T02:36:10.985ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
60952025-01-15T02:36:10.985ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 481.264087ms iteration = 10 status = {"generation":1}
60962025-01-15T02:36:10.991ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
60972025-01-15T02:36:10.991ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
60982025-01-15T02:36:10.991ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
60992025-01-15T02:36:10.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61002025-01-15T02:36:10.993ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61012025-01-15T02:36:10.997ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61022025-01-15T02:36:10.997ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61032025-01-15T02:36:10.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61042025-01-15T02:36:10.999ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
61052025-01-15T02:36:10.999ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
61062025-01-15T02:36:10.999ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
61072025-01-15T02:36:10.999ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 495.188282ms iteration = 10 status = {"generation":2}
61082025-01-15T02:36:11.012ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
61092025-01-15T02:36:11.012ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 507.71373ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
61102025-01-15T02:36:11.018ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
61112025-01-15T02:36:11.018ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbf36b74-ce72-4905-a509-f3fca689bb3b", "content-length": "1", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} })
61122025-01-15T02:36:11.019ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
61132025-01-15T02:36:11.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
61142025-01-15T02:36:11.021ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61152025-01-15T02:36:11.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61162025-01-15T02:36:11.021ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 517.886755ms iteration = 19 status = null
61172025-01-15T02:36:11.022ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4df2fe26-ba5b-4858-9516-c0133e7fe242", "content-length": "372", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
61182025-01-15T02:36:11.022ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61192025-01-15T02:36:11.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61202025-01-15T02:36:11.023ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61212025-01-15T02:36:11.025ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61222025-01-15T02:36:11.025ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61232025-01-15T02:36:11.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61242025-01-15T02:36:11.030ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
61252025-01-15T02:36:11.034ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 530.474224ms iteration = 19 status = {}
61262025-01-15T02:36:11.061ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61272025-01-15T02:36:11.063ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61282025-01-15T02:36:11.063ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61292025-01-15T02:36:11.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61302025-01-15T02:36:11.070ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
61312025-01-15T02:36:11.070ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
61322025-01-15T02:36:11.070ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "119c7472-6f3f-48be-a67f-e49bca69a2e7", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
61332025-01-15T02:36:11.070ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61342025-01-15T02:36:11.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61352025-01-15T02:36:11.073ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61362025-01-15T02:36:11.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61372025-01-15T02:36:11.076ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61382025-01-15T02:36:11.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61392025-01-15T02:36:11.079ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
61402025-01-15T02:36:11.079ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
61412025-01-15T02:36:11.080ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ca80fe26-6da6-48d2-b829-4a57af0af4bc", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
61422025-01-15T02:36:11.082ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 578.336136ms iteration = 19 status = {}
61432025-01-15T02:36:11.101ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
61442025-01-15T02:36:11.101ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f853f11d-6979-47f2-a72d-2680271399a6", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61452025-01-15T02:36:11.101ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61462025-01-15T02:36:11.103ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5a2ae28b-e48e-4fd1-a860-ac9b665a51c7", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61472025-01-15T02:36:11.103ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61482025-01-15T02:36:11.105ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "c09accc8-15f0-4ae5-baf1-010d27bd38fe", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61492025-01-15T02:36:11.105ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61502025-01-15T02:36:11.107ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
61512025-01-15T02:36:11.107ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 604.06646ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
61522025-01-15T02:36:11.115ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61532025-01-15T02:36:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61542025-01-15T02:36:11.118ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
61552025-01-15T02:36:11.118ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61562025-01-15T02:36:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61572025-01-15T02:36:11.123ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
61582025-01-15T02:36:11.124ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8371d08f-4b62-44a7-9db2-df26c715d538", "content-length": "691", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
61592025-01-15T02:36:11.124ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
61602025-01-15T02:36:11.124ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
61612025-01-15T02:36:11.124ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
61622025-01-15T02:36:11.125ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 620.899084ms iteration = 19 status = {}
61632025-01-15T02:36:11.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
61642025-01-15T02:36:11.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f7a54a4-0772-4ffd-8356-34923e668cc7", "content-length": "134", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} })
61652025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
61662025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
61672025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
61682025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
61692025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61702025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61712025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61722025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
61732025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61742025-01-15T02:36:11.133ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
61752025-01-15T02:36:11.134ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
61762025-01-15T02:36:11.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61772025-01-15T02:36:11.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61782025-01-15T02:36:11.135ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
61792025-01-15T02:36:11.136ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4f9f76f-245c-46c4-b00c-cababe6385c6", "content-length": "2", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} })
61802025-01-15T02:36:11.136ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
61812025-01-15T02:36:11.136ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "49062bf4-3163-492a-9220-c66fd7f22c37", "content-length": "52", "date": "Wed, 15 Jan 2025 02:36:11 GMT"} })
61822025-01-15T02:36:11.136ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
61832025-01-15T02:36:11.136ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
61842025-01-15T02:36:11.230ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "90bc75d0-5916-4573-9cbb-a3ed577639d4", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
61852025-01-15T02:36:11.230ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
61862025-01-15T02:36:11.230ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
61872025-01-15T02:36:11.230ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "22f7b3a7-3775-454d-9eec-40e30693f64c", "content-length": "682", "date": "Wed, 15 Jan 2025 02:36:10 GMT"} })
61882025-01-15T02:36:11.230ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61892025-01-15T02:36:11.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61902025-01-15T02:36:11.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61912025-01-15T02:36:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61922025-01-15T02:36:11.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61932025-01-15T02:36:11.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61942025-01-15T02:36:11.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61952025-01-15T02:36:11.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61962025-01-15T02:36:11.246ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 742.215358ms iteration = 19 status = {}
61972025-01-15T02:36:11.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 08b2aea3-1251-4944-8669-fe0df61cdaed roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
61982025-01-15T02:36:11.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 08b2aea3-1251-4944-8669-fe0df61cdaed resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
61992025-01-15T02:36:11.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2341 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 08b2aea3-1251-4944-8669-fe0df61cdaed response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62002025-01-15T02:36:12.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a013c294-505f-47ca-8d83-0de3a8db2161 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62012025-01-15T02:36:12.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a013c294-505f-47ca-8d83-0de3a8db2161 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62022025-01-15T02:36:12.467ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2111 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a013c294-505f-47ca-8d83-0de3a8db2161 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62032025-01-15T02:36:13.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4e8ba3d-8ea2-4fd2-b3ff-1fc9bc117f08 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62042025-01-15T02:36:13.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4e8ba3d-8ea2-4fd2-b3ff-1fc9bc117f08 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62052025-01-15T02:36:13.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2122 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b4e8ba3d-8ea2-4fd2-b3ff-1fc9bc117f08 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62062025-01-15T02:36:14.511ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7f71adf-1784-4d1a-a438-152913538460 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62072025-01-15T02:36:14.511ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7f71adf-1784-4d1a-a438-152913538460 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62082025-01-15T02:36:14.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2668 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7f71adf-1784-4d1a-a438-152913538460 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62092025-01-15T02:36:15.532ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1de47971-247d-48bd-a01b-ba29dd820eed roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62102025-01-15T02:36:15.533ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1de47971-247d-48bd-a01b-ba29dd820eed resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62112025-01-15T02:36:15.535ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2355 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1de47971-247d-48bd-a01b-ba29dd820eed response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62122025-01-15T02:36:15.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 89d8ff70-24bb-4121-9ef7-6afd4012537d roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
62132025-01-15T02:36:15.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 89d8ff70-24bb-4121-9ef7-6afd4012537d resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
62142025-01-15T02:36:15.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2845 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 89d8ff70-24bb-4121-9ef7-6afd4012537d response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
62152025-01-15T02:36:15.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 01289d98-b38a-4c07-bc56-e135c945c237 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
62162025-01-15T02:36:15.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 01289d98-b38a-4c07-bc56-e135c945c237 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
62172025-01-15T02:36:15.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1772 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 01289d98-b38a-4c07-bc56-e135c945c237 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
62182025-01-15T02:36:15.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 517 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = ce634cc7-8628-4e63-a397-f153335ba562 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
62192025-01-15T02:36:16.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 14d51307-de5c-4be1-b61c-515e59036972 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62202025-01-15T02:36:16.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 14d51307-de5c-4be1-b61c-515e59036972 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62212025-01-15T02:36:16.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2243 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 14d51307-de5c-4be1-b61c-515e59036972 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62222025-01-15T02:36:17.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ee9f198-d8fb-4b2b-a12f-b5af53877a5e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62232025-01-15T02:36:17.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ee9f198-d8fb-4b2b-a12f-b5af53877a5e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62242025-01-15T02:36:17.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6ee9f198-d8fb-4b2b-a12f-b5af53877a5e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62252025-01-15T02:36:18.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = defb0d6f-6dec-4cdf-a4fa-c45bdb02237b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62262025-01-15T02:36:18.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = defb0d6f-6dec-4cdf-a4fa-c45bdb02237b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62272025-01-15T02:36:18.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2074 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = defb0d6f-6dec-4cdf-a4fa-c45bdb02237b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62282025-01-15T02:36:19.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1e4b631-62a9-4f63-90bb-5cb226b229cc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62292025-01-15T02:36:19.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1e4b631-62a9-4f63-90bb-5cb226b229cc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62302025-01-15T02:36:19.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6901 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1e4b631-62a9-4f63-90bb-5cb226b229cc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62312025-01-15T02:36:20.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
62322025-01-15T02:36:20.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
62332025-01-15T02:36:20.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62342025-01-15T02:36:20.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62352025-01-15T02:36:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62362025-01-15T02:36:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62372025-01-15T02:36:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62382025-01-15T02:36:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62392025-01-15T02:36:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62402025-01-15T02:36:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62412025-01-15T02:36:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62422025-01-15T02:36:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62432025-01-15T02:36:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62442025-01-15T02:36:20.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
62452025-01-15T02:36:20.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62462025-01-15T02:36:20.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62472025-01-15T02:36:20.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
62482025-01-15T02:36:20.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.753524ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
62492025-01-15T02:36:20.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62502025-01-15T02:36:20.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62512025-01-15T02:36:20.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.082547ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62522025-01-15T02:36:20.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 20b6d6d9-f341-4118-9ebd-a0aca133164b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62532025-01-15T02:36:20.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 20b6d6d9-f341-4118-9ebd-a0aca133164b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62542025-01-15T02:36:20.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2082 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 20b6d6d9-f341-4118-9ebd-a0aca133164b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62552025-01-15T02:36:21.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 07a7cf45-2712-41e0-a823-47d4ab919208 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62562025-01-15T02:36:21.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 07a7cf45-2712-41e0-a823-47d4ab919208 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62572025-01-15T02:36:21.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2050 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 07a7cf45-2712-41e0-a823-47d4ab919208 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62582025-01-15T02:36:22.693ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0a3f20c2-27a1-4cc5-a4b3-d32fb10935c8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62592025-01-15T02:36:22.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0a3f20c2-27a1-4cc5-a4b3-d32fb10935c8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62602025-01-15T02:36:22.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0a3f20c2-27a1-4cc5-a4b3-d32fb10935c8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62612025-01-15T02:36:23.714ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f5bfbad-7354-4343-96f4-251b4320ad96 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62622025-01-15T02:36:23.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f5bfbad-7354-4343-96f4-251b4320ad96 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62632025-01-15T02:36:23.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2074 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f5bfbad-7354-4343-96f4-251b4320ad96 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62642025-01-15T02:36:24.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0553f18a-76dd-4906-a2b5-5929490749a7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62652025-01-15T02:36:24.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0553f18a-76dd-4906-a2b5-5929490749a7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62662025-01-15T02:36:24.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1950 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0553f18a-76dd-4906-a2b5-5929490749a7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62672025-01-15T02:36:25.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 487 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 4404baf8-feca-470e-9b0d-bd90ff7e195f response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
62682025-01-15T02:36:25.757ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be421b6b-f113-4fdf-b7d2-232d3ae92542 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62692025-01-15T02:36:25.757ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be421b6b-f113-4fdf-b7d2-232d3ae92542 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62702025-01-15T02:36:25.759ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2403 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = be421b6b-f113-4fdf-b7d2-232d3ae92542 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62712025-01-15T02:36:26.779ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d892c99-eb3e-42c8-a6a2-f22c1b9c1be4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62722025-01-15T02:36:26.779ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d892c99-eb3e-42c8-a6a2-f22c1b9c1be4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62732025-01-15T02:36:26.781ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1827 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1d892c99-eb3e-42c8-a6a2-f22c1b9c1be4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62742025-01-15T02:36:27.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88c5980f-f291-4ccc-bb8e-c6949e16bf99 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62752025-01-15T02:36:27.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88c5980f-f291-4ccc-bb8e-c6949e16bf99 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62762025-01-15T02:36:27.803ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2129 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88c5980f-f291-4ccc-bb8e-c6949e16bf99 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62772025-01-15T02:36:28.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 758d5e53-590d-4363-8bee-c24314311de4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62782025-01-15T02:36:28.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 758d5e53-590d-4363-8bee-c24314311de4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62792025-01-15T02:36:28.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2224 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 758d5e53-590d-4363-8bee-c24314311de4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62802025-01-15T02:36:29.845ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ddcc73e0-cec5-4880-9659-8647518e6ec6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62812025-01-15T02:36:29.846ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ddcc73e0-cec5-4880-9659-8647518e6ec6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62822025-01-15T02:36:29.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1955 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ddcc73e0-cec5-4880-9659-8647518e6ec6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
62832025-01-15T02:36:30.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
62842025-01-15T02:36:30.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
62852025-01-15T02:36:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62862025-01-15T02:36:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62872025-01-15T02:36:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62882025-01-15T02:36:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62892025-01-15T02:36:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62902025-01-15T02:36:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62912025-01-15T02:36:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62922025-01-15T02:36:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62932025-01-15T02:36:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62942025-01-15T02:36:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62952025-01-15T02:36:30.536ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62962025-01-15T02:36:30.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62972025-01-15T02:36:30.537ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.996541ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62982025-01-15T02:36:30.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62992025-01-15T02:36:30.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
63002025-01-15T02:36:30.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63012025-01-15T02:36:30.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63022025-01-15T02:36:30.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
63032025-01-15T02:36:30.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.741843ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
63042025-01-15T02:36:30.868ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = adccebe3-51fe-4192-843a-af2354dd945a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63052025-01-15T02:36:30.869ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = adccebe3-51fe-4192-843a-af2354dd945a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63062025-01-15T02:36:30.870ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2392 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = adccebe3-51fe-4192-843a-af2354dd945a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63072025-01-15T02:36:31.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e1486ca1-5dcb-4b06-9142-72c87de8af2b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63082025-01-15T02:36:31.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e1486ca1-5dcb-4b06-9142-72c87de8af2b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63092025-01-15T02:36:31.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2054 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e1486ca1-5dcb-4b06-9142-72c87de8af2b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63102025-01-15T02:36:32.914ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f4e8dad5-e2a4-4f0a-bb85-fe50e962c080 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63112025-01-15T02:36:32.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f4e8dad5-e2a4-4f0a-bb85-fe50e962c080 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63122025-01-15T02:36:32.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1993 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f4e8dad5-e2a4-4f0a-bb85-fe50e962c080 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63132025-01-15T02:36:33.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29ad41c8-9d02-40af-9094-8aa6f482be5f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63142025-01-15T02:36:33.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29ad41c8-9d02-40af-9094-8aa6f482be5f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63152025-01-15T02:36:33.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1960 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29ad41c8-9d02-40af-9094-8aa6f482be5f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63162025-01-15T02:36:34.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0cf6ee68-eb2e-4c9f-b9e8-56e77a8483b0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63172025-01-15T02:36:34.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0cf6ee68-eb2e-4c9f-b9e8-56e77a8483b0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63182025-01-15T02:36:34.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2582 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0cf6ee68-eb2e-4c9f-b9e8-56e77a8483b0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63192025-01-15T02:36:35.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 593 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 677058e2-423a-46fc-bc31-0189d9c03666 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
63202025-01-15T02:36:35.979ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7386803-665c-4ab0-ad3e-d2c17a891552 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63212025-01-15T02:36:35.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7386803-665c-4ab0-ad3e-d2c17a891552 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63222025-01-15T02:36:35.981ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2570 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7386803-665c-4ab0-ad3e-d2c17a891552 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63232025-01-15T02:36:37.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f09bdc01-34b7-4d8a-aae4-48ee7cea43ff roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63242025-01-15T02:36:37.003ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f09bdc01-34b7-4d8a-aae4-48ee7cea43ff resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63252025-01-15T02:36:37.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1973 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f09bdc01-34b7-4d8a-aae4-48ee7cea43ff response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63262025-01-15T02:36:38.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 98594bb9-38f3-490e-b423-aef215fa9497 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63272025-01-15T02:36:38.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 98594bb9-38f3-490e-b423-aef215fa9497 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63282025-01-15T02:36:38.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1875 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 98594bb9-38f3-490e-b423-aef215fa9497 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63292025-01-15T02:36:39.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32eb103a-ff5e-4454-bb3a-f796905b6078 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63302025-01-15T02:36:39.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32eb103a-ff5e-4454-bb3a-f796905b6078 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63312025-01-15T02:36:39.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2086 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 32eb103a-ff5e-4454-bb3a-f796905b6078 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63322025-01-15T02:36:40.068ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d1cb2698-40d2-4bfa-852d-0ecfe1f4f5cf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63332025-01-15T02:36:40.068ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d1cb2698-40d2-4bfa-852d-0ecfe1f4f5cf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63342025-01-15T02:36:40.074ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d1cb2698-40d2-4bfa-852d-0ecfe1f4f5cf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
63352025-01-15T02:36:40.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
63362025-01-15T02:36:40.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
63372025-01-15T02:36:40.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
63382025-01-15T02:36:40.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
63392025-01-15T02:36:40.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
63402025-01-15T02:36:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
63412025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
63422025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
63432025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
63442025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
63452025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63462025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
63472025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63482025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63492025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
63502025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63512025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63522025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
63532025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63542025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
63552025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63562025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
63572025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
63582025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63592025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
63602025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63612025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
63622025-01-15T02:36:40.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
63632025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63642025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
63652025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63662025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63672025-01-15T02:36:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
63682025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63692025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63702025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63712025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63722025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63732025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63742025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63752025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63762025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63772025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
63782025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
63792025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63802025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
63812025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
63822025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
63832025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
63842025-01-15T02:36:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63852025-01-15T02:36:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
63862025-01-15T02:36:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
63872025-01-15T02:36:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63882025-01-15T02:36:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63892025-01-15T02:36:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3887, tv_nsec: 344230293 } })
63902025-01-15T02:36:40.507ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.395062ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
63912025-01-15T02:36:40.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3887, tv_nsec: 344400268 } })
63922025-01-15T02:36:40.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
63932025-01-15T02:36:40.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
63942025-01-15T02:36:40.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63952025-01-15T02:36:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63962025-01-15T02:36:40.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63972025-01-15T02:36:40.509ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63982025-01-15T02:36:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
63992025-01-15T02:36:40.510ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64002025-01-15T02:36:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64012025-01-15T02:36:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64022025-01-15T02:36:40.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64032025-01-15T02:36:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
64042025-01-15T02:36:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
64052025-01-15T02:36:40.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64062025-01-15T02:36:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64072025-01-15T02:36:40.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64082025-01-15T02:36:40.512ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64092025-01-15T02:36:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
64102025-01-15T02:36:40.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64112025-01-15T02:36:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64122025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64132025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64142025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
64152025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64162025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64172025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64182025-01-15T02:36:40.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64192025-01-15T02:36:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
64202025-01-15T02:36:40.516ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64212025-01-15T02:36:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64222025-01-15T02:36:40.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64232025-01-15T02:36:40.519ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64242025-01-15T02:36:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64252025-01-15T02:36:40.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64262025-01-15T02:36:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64272025-01-15T02:36:40.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64282025-01-15T02:36:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64292025-01-15T02:36:40.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64302025-01-15T02:36:40.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64312025-01-15T02:36:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64322025-01-15T02:36:40.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3887, tv_nsec: 359212277 } })
64332025-01-15T02:36:40.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
64342025-01-15T02:36:40.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64352025-01-15T02:36:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
64362025-01-15T02:36:40.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64372025-01-15T02:36:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64382025-01-15T02:36:40.546ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64392025-01-15T02:36:40.546ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
64402025-01-15T02:36:40.546ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64412025-01-15T02:36:40.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64422025-01-15T02:36:40.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
64432025-01-15T02:36:40.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64442025-01-15T02:36:40.555ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64452025-01-15T02:36:40.555ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 50.152523ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
64462025-01-15T02:36:40.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64472025-01-15T02:36:40.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
64482025-01-15T02:36:40.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
64492025-01-15T02:36:40.560ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.288155ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
64502025-01-15T02:36:40.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64512025-01-15T02:36:40.561ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64522025-01-15T02:36:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64532025-01-15T02:36:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64542025-01-15T02:36:40.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64552025-01-15T02:36:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
64562025-01-15T02:36:40.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64572025-01-15T02:36:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64582025-01-15T02:36:40.574ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
64592025-01-15T02:36:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
64602025-01-15T02:36:40.578ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
64612025-01-15T02:36:40.578ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 74.253404ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
64622025-01-15T02:36:40.579ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64632025-01-15T02:36:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64642025-01-15T02:36:40.589ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
64652025-01-15T02:36:40.589ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.85982ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
64662025-01-15T02:36:40.591ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.177843ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64672025-01-15T02:36:40.594ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
64682025-01-15T02:36:40.594ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64692025-01-15T02:36:40.594ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb11abdb-3462-471b-a91a-4cfe822a647d", "content-length": "26", "date": "Wed, 15 Jan 2025 02:36:39 GMT"} })
64702025-01-15T02:36:40.594ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64712025-01-15T02:36:40.596ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64722025-01-15T02:36:40.601ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64732025-01-15T02:36:40.601ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
64742025-01-15T02:36:40.601ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64752025-01-15T02:36:40.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
64762025-01-15T02:36:40.604ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
64772025-01-15T02:36:40.604ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64782025-01-15T02:36:40.604ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d72219f3-5c64-478c-b35d-0992c971d913", "content-length": "26", "date": "Wed, 15 Jan 2025 02:36:39 GMT"} })
64792025-01-15T02:36:40.604ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64802025-01-15T02:36:40.605ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 99.383541ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
64812025-01-15T02:36:40.605ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 100.791419ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
64822025-01-15T02:36:40.606ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64832025-01-15T02:36:40.610ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64842025-01-15T02:36:40.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
64852025-01-15T02:36:40.612ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64862025-01-15T02:36:40.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64872025-01-15T02:36:40.614ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
64882025-01-15T02:36:40.614ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 109.858805ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
64892025-01-15T02:36:40.615ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64902025-01-15T02:36:40.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
64912025-01-15T02:36:40.617ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64922025-01-15T02:36:40.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64932025-01-15T02:36:40.620ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64942025-01-15T02:36:40.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64952025-01-15T02:36:40.622ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 117.390618ms iteration = 20 status = 0
64962025-01-15T02:36:40.625ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 119.935896ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
64972025-01-15T02:36:40.643ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 138.706773ms iteration = 22 status = {"physical_disks_added":0}
64982025-01-15T02:36:40.644ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64992025-01-15T02:36:40.644ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
65002025-01-15T02:36:40.644ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 139.484194ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
65012025-01-15T02:36:40.645ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65022025-01-15T02:36:40.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65032025-01-15T02:36:40.695ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
65042025-01-15T02:36:40.695ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65052025-01-15T02:36:40.696ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3219d45b-95e5-4668-8df9-f061eaf34cdf", "content-length": "26", "date": "Wed, 15 Jan 2025 02:36:39 GMT"} })
65062025-01-15T02:36:40.696ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65072025-01-15T02:36:40.697ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
65082025-01-15T02:36:40.728ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
65092025-01-15T02:36:40.728ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65102025-01-15T02:36:40.729ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ed12758-8789-4cf0-9dab-d57c2b1595f5", "content-length": "372", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65112025-01-15T02:36:40.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65122025-01-15T02:36:40.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65132025-01-15T02:36:40.738ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65142025-01-15T02:36:40.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65152025-01-15T02:36:40.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65162025-01-15T02:36:40.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
65172025-01-15T02:36:40.741ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65182025-01-15T02:36:40.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65192025-01-15T02:36:40.742ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
65202025-01-15T02:36:40.742ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "28a16bab-d3f3-4f5a-be28-3729bcc788de", "content-length": "1", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65212025-01-15T02:36:40.743ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65222025-01-15T02:36:40.744ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
65232025-01-15T02:36:40.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
65242025-01-15T02:36:40.745ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
65252025-01-15T02:36:40.745ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65262025-01-15T02:36:40.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65272025-01-15T02:36:40.745ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 241.970429ms iteration = 20 status = null
65282025-01-15T02:36:40.746ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 243.172381ms iteration = 20 status = {}
65292025-01-15T02:36:40.771ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
65302025-01-15T02:36:40.771ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
65312025-01-15T02:36:40.859ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "514974c1-beb9-46ef-918a-337cc176a0e1", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65322025-01-15T02:36:40.859ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65332025-01-15T02:36:40.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65342025-01-15T02:36:40.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65352025-01-15T02:36:40.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65362025-01-15T02:36:40.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65372025-01-15T02:36:40.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65382025-01-15T02:36:40.863ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
65392025-01-15T02:36:40.863ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
65402025-01-15T02:36:40.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65412025-01-15T02:36:40.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65422025-01-15T02:36:40.864ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c511afa-e63d-4ccc-aa1c-6aca347807a4", "content-length": "691", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65432025-01-15T02:36:40.864ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
65442025-01-15T02:36:40.864ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
65452025-01-15T02:36:40.864ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
65462025-01-15T02:36:40.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65472025-01-15T02:36:40.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65482025-01-15T02:36:40.866ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 361.621248ms iteration = 20 status = {}
65492025-01-15T02:36:40.868ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
65502025-01-15T02:36:40.868ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65512025-01-15T02:36:40.868ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7663a6b5-ad75-4a07-b351-51ba6bad89e5", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65522025-01-15T02:36:40.870ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 366.108249ms iteration = 20 status = {}
65532025-01-15T02:36:40.874ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
65542025-01-15T02:36:40.874ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d66a4484-28fa-4e03-bcd9-78fe10bce09a", "content-length": "134", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65552025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
65562025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
65572025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
65582025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
65592025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65602025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65612025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65622025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
65632025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65642025-01-15T02:36:40.874ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
65652025-01-15T02:36:40.875ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
65662025-01-15T02:36:40.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65672025-01-15T02:36:40.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65682025-01-15T02:36:40.877ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
65692025-01-15T02:36:40.877ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fcb069f-566b-4b01-a7f4-4cb940ca5ed6", "content-length": "2", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65702025-01-15T02:36:40.877ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
65712025-01-15T02:36:40.877ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "6175b89e-ad2e-4f2d-828b-d15ab0a7c331", "content-length": "52", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65722025-01-15T02:36:40.877ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
65732025-01-15T02:36:40.877ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
65742025-01-15T02:36:40.970ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "be0cd398-7603-4a63-8585-bf8f90eafb4c", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65752025-01-15T02:36:40.970ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
65762025-01-15T02:36:40.970ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
65772025-01-15T02:36:40.970ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "26356018-86cb-4d3c-89c9-21e25cae1f63", "content-length": "682", "date": "Wed, 15 Jan 2025 02:36:40 GMT"} })
65782025-01-15T02:36:40.970ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65792025-01-15T02:36:40.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65802025-01-15T02:36:40.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65812025-01-15T02:36:40.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65822025-01-15T02:36:40.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65832025-01-15T02:36:40.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65842025-01-15T02:36:40.978ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65852025-01-15T02:36:40.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65862025-01-15T02:36:40.985ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 480.889376ms iteration = 20 status = {}
65872025-01-15T02:36:41.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 86b86a95-a622-401e-adb9-e2caf3b408ad roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65882025-01-15T02:36:41.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 86b86a95-a622-401e-adb9-e2caf3b408ad resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65892025-01-15T02:36:41.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2303 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 86b86a95-a622-401e-adb9-e2caf3b408ad response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65902025-01-15T02:36:42.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a51b7f89-cded-4adc-a071-3a3ea0e89f95 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65912025-01-15T02:36:42.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a51b7f89-cded-4adc-a071-3a3ea0e89f95 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65922025-01-15T02:36:42.119ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2073 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a51b7f89-cded-4adc-a071-3a3ea0e89f95 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65932025-01-15T02:36:43.138ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b551b998-8099-4287-b845-a1a486abab77 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65942025-01-15T02:36:43.139ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b551b998-8099-4287-b845-a1a486abab77 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65952025-01-15T02:36:43.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2411 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b551b998-8099-4287-b845-a1a486abab77 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65962025-01-15T02:36:44.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8549b854-96a6-4563-ae5c-bc4fcf999dc8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65972025-01-15T02:36:44.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8549b854-96a6-4563-ae5c-bc4fcf999dc8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65982025-01-15T02:36:44.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1893 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8549b854-96a6-4563-ae5c-bc4fcf999dc8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
65992025-01-15T02:36:45.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 59a0e6aa-9d49-4792-af3d-d3354defef08 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66002025-01-15T02:36:45.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 59a0e6aa-9d49-4792-af3d-d3354defef08 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66012025-01-15T02:36:45.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2021 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 59a0e6aa-9d49-4792-af3d-d3354defef08 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66022025-01-15T02:36:45.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 426 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = f365a529-1130-4248-8646-5d011f73e77b response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
66032025-01-15T02:36:46.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a26994ca-e659-4fdf-9844-4a18d4b59e6c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66042025-01-15T02:36:46.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a26994ca-e659-4fdf-9844-4a18d4b59e6c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66052025-01-15T02:36:46.205ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1968 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a26994ca-e659-4fdf-9844-4a18d4b59e6c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66062025-01-15T02:36:47.224ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 99b0bd95-a823-4ab6-9e09-7fd9ff72c5c6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66072025-01-15T02:36:47.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 99b0bd95-a823-4ab6-9e09-7fd9ff72c5c6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66082025-01-15T02:36:47.227ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2293 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 99b0bd95-a823-4ab6-9e09-7fd9ff72c5c6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66092025-01-15T02:36:48.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cba02fe-c86f-45dd-9b44-2b1e06b40e36 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66102025-01-15T02:36:48.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cba02fe-c86f-45dd-9b44-2b1e06b40e36 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66112025-01-15T02:36:48.248ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2011 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cba02fe-c86f-45dd-9b44-2b1e06b40e36 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66122025-01-15T02:36:49.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d69c828-5241-4969-a8f1-d4606a2896c6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66132025-01-15T02:36:49.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d69c828-5241-4969-a8f1-d4606a2896c6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66142025-01-15T02:36:49.271ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1945 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d69c828-5241-4969-a8f1-d4606a2896c6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66152025-01-15T02:36:50.290ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 21749c61-5315-429a-9653-3d376bf8ff74 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66162025-01-15T02:36:50.290ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 21749c61-5315-429a-9653-3d376bf8ff74 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66172025-01-15T02:36:50.292ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2229 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 21749c61-5315-429a-9653-3d376bf8ff74 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66182025-01-15T02:36:50.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
66192025-01-15T02:36:50.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
66202025-01-15T02:36:50.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66212025-01-15T02:36:50.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66222025-01-15T02:36:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66232025-01-15T02:36:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66242025-01-15T02:36:50.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66252025-01-15T02:36:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66262025-01-15T02:36:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66272025-01-15T02:36:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66282025-01-15T02:36:50.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66292025-01-15T02:36:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66302025-01-15T02:36:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66312025-01-15T02:36:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66322025-01-15T02:36:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66332025-01-15T02:36:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
66342025-01-15T02:36:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66352025-01-15T02:36:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66362025-01-15T02:36:50.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
66372025-01-15T02:36:50.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.538121ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
66382025-01-15T02:36:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.59882ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
66392025-01-15T02:36:51.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 433eb8fa-f5e0-4148-88a9-f21431a7b4c7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66402025-01-15T02:36:51.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 433eb8fa-f5e0-4148-88a9-f21431a7b4c7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66412025-01-15T02:36:51.315ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 433eb8fa-f5e0-4148-88a9-f21431a7b4c7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66422025-01-15T02:36:52.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1400035-6574-4b64-a428-ba51729975ae roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66432025-01-15T02:36:52.334ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1400035-6574-4b64-a428-ba51729975ae resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66442025-01-15T02:36:52.336ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2355 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1400035-6574-4b64-a428-ba51729975ae response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66452025-01-15T02:36:53.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89801b0c-fe31-415a-8716-cfbb1d712e3d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66462025-01-15T02:36:53.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89801b0c-fe31-415a-8716-cfbb1d712e3d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66472025-01-15T02:36:53.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2272 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89801b0c-fe31-415a-8716-cfbb1d712e3d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66482025-01-15T02:36:54.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fdc2e8bd-bd04-4577-83ed-28104a90613c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66492025-01-15T02:36:54.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fdc2e8bd-bd04-4577-83ed-28104a90613c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66502025-01-15T02:36:54.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2499 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fdc2e8bd-bd04-4577-83ed-28104a90613c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66512025-01-15T02:36:55.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8bf5fed9-024a-48b3-973c-abad22dba375 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66522025-01-15T02:36:55.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8bf5fed9-024a-48b3-973c-abad22dba375 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66532025-01-15T02:36:55.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2479 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8bf5fed9-024a-48b3-973c-abad22dba375 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66542025-01-15T02:36:55.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 506 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = c41f3ad2-a1a6-4f3a-b4a3-e4949d8c0b47 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
66552025-01-15T02:36:56.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7872469e-20df-437c-a491-f2cbe19f5ce0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66562025-01-15T02:36:56.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7872469e-20df-437c-a491-f2cbe19f5ce0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66572025-01-15T02:36:56.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2036 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7872469e-20df-437c-a491-f2cbe19f5ce0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66582025-01-15T02:36:57.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e139f9c-1511-4baf-9994-af95aabb433f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66592025-01-15T02:36:57.449ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e139f9c-1511-4baf-9994-af95aabb433f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66602025-01-15T02:36:57.451ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2108 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e139f9c-1511-4baf-9994-af95aabb433f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66612025-01-15T02:36:58.470ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4721a2d6-9da9-41bb-9f1a-1c06f1a2d44d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66622025-01-15T02:36:58.470ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4721a2d6-9da9-41bb-9f1a-1c06f1a2d44d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66632025-01-15T02:36:58.472ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2237 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4721a2d6-9da9-41bb-9f1a-1c06f1a2d44d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66642025-01-15T02:36:59.492ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5db504c-0603-4e95-aefc-ec2fd01f7cc3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66652025-01-15T02:36:59.492ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5db504c-0603-4e95-aefc-ec2fd01f7cc3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66662025-01-15T02:36:59.494ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2143 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f5db504c-0603-4e95-aefc-ec2fd01f7cc3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66672025-01-15T02:37:00.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
66682025-01-15T02:37:00.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
66692025-01-15T02:37:00.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66702025-01-15T02:37:00.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66712025-01-15T02:37:00.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66722025-01-15T02:37:00.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66732025-01-15T02:37:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66742025-01-15T02:37:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66752025-01-15T02:37:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66762025-01-15T02:37:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66772025-01-15T02:37:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66782025-01-15T02:37:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66792025-01-15T02:37:00.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66802025-01-15T02:37:00.513ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8af9dff6-a650-4260-b087-d5428bedb0db roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66812025-01-15T02:37:00.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8af9dff6-a650-4260-b087-d5428bedb0db resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66822025-01-15T02:37:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
66832025-01-15T02:37:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66842025-01-15T02:37:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66852025-01-15T02:37:00.514ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1876 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8af9dff6-a650-4260-b087-d5428bedb0db response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66862025-01-15T02:37:00.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
66872025-01-15T02:37:00.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.824588ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
66882025-01-15T02:37:00.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66892025-01-15T02:37:00.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66902025-01-15T02:37:00.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.947008ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
66912025-01-15T02:37:01.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = afc57693-1811-4480-b8b2-699ededba4a1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66922025-01-15T02:37:01.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = afc57693-1811-4480-b8b2-699ededba4a1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66932025-01-15T02:37:01.536ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1934 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = afc57693-1811-4480-b8b2-699ededba4a1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66942025-01-15T02:37:02.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c308fd8f-22ea-41ed-9de7-c2200cf666ba roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66952025-01-15T02:37:02.558ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c308fd8f-22ea-41ed-9de7-c2200cf666ba resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66962025-01-15T02:37:02.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1994 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c308fd8f-22ea-41ed-9de7-c2200cf666ba response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66972025-01-15T02:37:03.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ee899b3f-6565-4c5e-a742-3abf0fc67675 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66982025-01-15T02:37:03.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ee899b3f-6565-4c5e-a742-3abf0fc67675 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
66992025-01-15T02:37:03.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2424 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ee899b3f-6565-4c5e-a742-3abf0fc67675 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67002025-01-15T02:37:04.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4952075c-9353-49b7-9ebb-67054f23399c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67012025-01-15T02:37:04.605ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4952075c-9353-49b7-9ebb-67054f23399c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67022025-01-15T02:37:04.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2062 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4952075c-9353-49b7-9ebb-67054f23399c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67032025-01-15T02:37:05.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84cdc189-9920-495f-979e-0254ffecacb7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67042025-01-15T02:37:05.626ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84cdc189-9920-495f-979e-0254ffecacb7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67052025-01-15T02:37:05.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2010 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 84cdc189-9920-495f-979e-0254ffecacb7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67062025-01-15T02:37:05.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 701 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 8ecc3a99-284e-449f-a330-e77dbe737139 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
67072025-01-15T02:37:06.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 68c42fff-f755-4fc3-865f-fdaf3e6973fe roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67082025-01-15T02:37:06.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 68c42fff-f755-4fc3-865f-fdaf3e6973fe resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67092025-01-15T02:37:06.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2006 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 68c42fff-f755-4fc3-865f-fdaf3e6973fe response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67102025-01-15T02:37:07.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a58400e-a49f-4c23-9b45-8f405b72c142 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67112025-01-15T02:37:07.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a58400e-a49f-4c23-9b45-8f405b72c142 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67122025-01-15T02:37:07.672ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a58400e-a49f-4c23-9b45-8f405b72c142 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67132025-01-15T02:37:08.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 084b77d0-3ee8-494d-aef6-24cdec19d93d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67142025-01-15T02:37:08.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 084b77d0-3ee8-494d-aef6-24cdec19d93d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67152025-01-15T02:37:08.694ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2241 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 084b77d0-3ee8-494d-aef6-24cdec19d93d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67162025-01-15T02:37:09.715ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40c7fda6-d73e-4039-b46e-c16087bb87af roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67172025-01-15T02:37:09.715ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40c7fda6-d73e-4039-b46e-c16087bb87af resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67182025-01-15T02:37:09.717ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2489 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 40c7fda6-d73e-4039-b46e-c16087bb87af response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
67192025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
67202025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
67212025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
67222025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
67232025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
67242025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
67252025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
67262025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
67272025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
67282025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
67292025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
67302025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
67312025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
67322025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
67332025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
67342025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
67352025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
67362025-01-15T02:37:10.503ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:27:10.503710866 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
67372025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
67382025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
67392025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
67402025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
67412025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
67422025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67432025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67442025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
67452025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67462025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67472025-01-15T02:37:10.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
67482025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
67492025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67502025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
67512025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67522025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
67532025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67542025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
67552025-01-15T02:37:10.504ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
67562025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
67572025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
67582025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67592025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
67602025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
67612025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
67622025-01-15T02:37:10.504ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
67632025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
67642025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
67652025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
67662025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
67672025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67682025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
67692025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
67702025-01-15T02:37:10.504ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
67712025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.11\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
67722025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
67732025-01-15T02:37:10.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
67742025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67752025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
67762025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
67772025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
67782025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
67792025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
67802025-01-15T02:37:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3917, tv_nsec: 341891170 } })
67812025-01-15T02:37:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3917, tv_nsec: 341922701 } })
67822025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67832025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67842025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67852025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
67862025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67872025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67882025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
67892025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67902025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
67912025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67922025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
67932025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67942025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
67952025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67962025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
67972025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.145751ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
67982025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
67992025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68002025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
68012025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
68022025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
68032025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
68042025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
68052025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
68062025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68072025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.390437ms iteration = 11 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
68082025-01-15T02:37:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
68092025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
68102025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
68112025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
68122025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
68132025-01-15T02:37:10.505ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
68142025-01-15T02:37:10.505ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
68152025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
68162025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
68172025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
68182025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68192025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
68202025-01-15T02:37:10.505ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
68212025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
68222025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
68232025-01-15T02:37:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
68242025-01-15T02:37:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.421312ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
68252025-01-15T02:37:10.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3917, tv_nsec: 344995784 } })
68262025-01-15T02:37:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
68272025-01-15T02:37:10.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
68282025-01-15T02:37:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
68292025-01-15T02:37:10.508ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:27:10.503710866 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
68302025-01-15T02:37:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.972924ms iteration = 11 status = {"expiration":"2025-01-15T02:27:10.503710866Z","pruned":[]}
68312025-01-15T02:37:10.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3917, tv_nsec: 346785282 } })
68322025-01-15T02:37:10.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
68332025-01-15T02:37:10.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68342025-01-15T02:37:10.509ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68352025-01-15T02:37:10.512ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68362025-01-15T02:37:10.512ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
68372025-01-15T02:37:10.512ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68382025-01-15T02:37:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68392025-01-15T02:37:10.512ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
68402025-01-15T02:37:10.512ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68412025-01-15T02:37:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
68422025-01-15T02:37:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
68432025-01-15T02:37:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68442025-01-15T02:37:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68452025-01-15T02:37:10.516ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68462025-01-15T02:37:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68472025-01-15T02:37:10.517ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68482025-01-15T02:37:10.517ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68492025-01-15T02:37:10.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68502025-01-15T02:37:10.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68512025-01-15T02:37:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
68522025-01-15T02:37:10.519ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
68532025-01-15T02:37:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
68542025-01-15T02:37:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
68552025-01-15T02:37:10.520ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68562025-01-15T02:37:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68572025-01-15T02:37:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
68582025-01-15T02:37:10.521ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68592025-01-15T02:37:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68602025-01-15T02:37:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
68612025-01-15T02:37:10.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68622025-01-15T02:37:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68632025-01-15T02:37:10.523ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
68642025-01-15T02:37:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
68652025-01-15T02:37:10.528ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 24.032059ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
68662025-01-15T02:37:10.536ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68672025-01-15T02:37:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
68682025-01-15T02:37:10.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68692025-01-15T02:37:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68702025-01-15T02:37:10.556ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68712025-01-15T02:37:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
68722025-01-15T02:37:10.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68732025-01-15T02:37:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68742025-01-15T02:37:10.562ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68752025-01-15T02:37:10.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
68762025-01-15T02:37:10.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
68772025-01-15T02:37:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
68782025-01-15T02:37:10.565ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68792025-01-15T02:37:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68802025-01-15T02:37:10.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68812025-01-15T02:37:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68822025-01-15T02:37:10.570ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
68832025-01-15T02:37:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
68842025-01-15T02:37:10.571ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.910585ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
68852025-01-15T02:37:10.577ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68862025-01-15T02:37:10.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68872025-01-15T02:37:10.579ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
68882025-01-15T02:37:10.579ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 74.424888ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
68892025-01-15T02:37:10.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68902025-01-15T02:37:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
68912025-01-15T02:37:10.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68922025-01-15T02:37:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68932025-01-15T02:37:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
68942025-01-15T02:37:10.582ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68952025-01-15T02:37:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
68962025-01-15T02:37:10.583ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
68972025-01-15T02:37:10.583ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68982025-01-15T02:37:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68992025-01-15T02:37:10.585ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
69002025-01-15T02:37:10.585ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
69012025-01-15T02:37:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 80.77288ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
69022025-01-15T02:37:10.587ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
69032025-01-15T02:37:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
69042025-01-15T02:37:10.588ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3917, tv_nsec: 425974419 } })
69052025-01-15T02:37:10.588ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
69062025-01-15T02:37:10.588ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69072025-01-15T02:37:10.589ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69082025-01-15T02:37:10.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69092025-01-15T02:37:10.590ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69102025-01-15T02:37:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69112025-01-15T02:37:10.591ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69122025-01-15T02:37:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69132025-01-15T02:37:10.591ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69142025-01-15T02:37:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69152025-01-15T02:37:10.593ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69162025-01-15T02:37:10.595ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 90.262085ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
69172025-01-15T02:37:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
69182025-01-15T02:37:10.596ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69192025-01-15T02:37:10.596ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
69202025-01-15T02:37:10.596ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69212025-01-15T02:37:10.596ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
69222025-01-15T02:37:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
69232025-01-15T02:37:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69242025-01-15T02:37:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
69252025-01-15T02:37:10.599ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69262025-01-15T02:37:10.600ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 96.564995ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69272025-01-15T02:37:10.600ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 95.737663ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
69282025-01-15T02:37:10.601ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69292025-01-15T02:37:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69302025-01-15T02:37:10.602ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
69312025-01-15T02:37:10.602ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69322025-01-15T02:37:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69332025-01-15T02:37:10.603ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
69342025-01-15T02:37:10.603ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 99.740132ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
69352025-01-15T02:37:10.605ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
69362025-01-15T02:37:10.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69372025-01-15T02:37:10.609ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
69382025-01-15T02:37:10.609ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
69392025-01-15T02:37:10.609ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 105.364953ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
69402025-01-15T02:37:10.611ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69412025-01-15T02:37:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69422025-01-15T02:37:10.619ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69432025-01-15T02:37:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69442025-01-15T02:37:10.623ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69452025-01-15T02:37:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
69462025-01-15T02:37:10.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69472025-01-15T02:37:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69482025-01-15T02:37:10.630ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
69492025-01-15T02:37:10.630ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 126.609166ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
69502025-01-15T02:37:10.633ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
69512025-01-15T02:37:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
69522025-01-15T02:37:10.635ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
69532025-01-15T02:37:10.636ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69542025-01-15T02:37:10.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69552025-01-15T02:37:10.640ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
69562025-01-15T02:37:10.640ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69572025-01-15T02:37:10.640ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e275e4d1-d906-4e8c-9ee7-196501c6460f", "content-length": "26", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
69582025-01-15T02:37:10.640ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69592025-01-15T02:37:10.641ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.724287ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
69602025-01-15T02:37:10.641ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
69612025-01-15T02:37:10.641ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
69622025-01-15T02:37:10.641ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
69632025-01-15T02:37:10.641ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69642025-01-15T02:37:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69652025-01-15T02:37:10.642ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
69662025-01-15T02:37:10.642ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69672025-01-15T02:37:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69682025-01-15T02:37:10.644ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
69692025-01-15T02:37:10.644ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
69702025-01-15T02:37:10.644ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
69712025-01-15T02:37:10.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69722025-01-15T02:37:10.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69732025-01-15T02:37:10.657ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69742025-01-15T02:37:10.657ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
69752025-01-15T02:37:10.657ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 153.494512ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
69762025-01-15T02:37:10.662ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 157.319245ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
69772025-01-15T02:37:10.662ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69782025-01-15T02:37:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
69792025-01-15T02:37:10.666ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
69802025-01-15T02:37:10.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
69812025-01-15T02:37:10.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
69822025-01-15T02:37:10.673ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
69832025-01-15T02:37:10.673ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69842025-01-15T02:37:10.673ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "78911df2-b370-4300-ad2a-60e7df207b50", "content-length": "26", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
69852025-01-15T02:37:10.673ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69862025-01-15T02:37:10.675ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
69872025-01-15T02:37:10.678ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69882025-01-15T02:37:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69892025-01-15T02:37:10.708ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 204.53227ms iteration = 23 status = {"physical_disks_added":0}
69902025-01-15T02:37:10.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a24e53f-b779-4b28-b17c-8fdf6aeda6bf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
69912025-01-15T02:37:10.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a24e53f-b779-4b28-b17c-8fdf6aeda6bf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
69922025-01-15T02:37:10.765ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
69932025-01-15T02:37:10.765ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69942025-01-15T02:37:10.766ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea7ccd43-b698-452c-975d-cb4fb8684807", "content-length": "26", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
69952025-01-15T02:37:10.766ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69962025-01-15T02:37:10.767ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
69972025-01-15T02:37:10.789ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
69982025-01-15T02:37:10.830ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
69992025-01-15T02:37:10.831ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "59bdadbb-bb6f-4f16-a910-44723fba6593", "content-length": "1", "date": "Wed, 15 Jan 2025 02:37:09 GMT"} })
70002025-01-15T02:37:10.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
70012025-01-15T02:37:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
70022025-01-15T02:37:10.855ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
70032025-01-15T02:37:10.855ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "484ffc16-a6cd-497c-9728-ab5717919efb", "date": "Wed, 15 Jan 2025 02:37:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70042025-01-15T02:37:10.855ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70052025-01-15T02:37:10.857ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "bbc4c3a4-6c48-4047-b498-2bfb9a31d9a5", "date": "Wed, 15 Jan 2025 02:37:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70062025-01-15T02:37:10.857ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70072025-01-15T02:37:10.858ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
70082025-01-15T02:37:10.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 355.242156ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
70092025-01-15T02:37:10.896ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
70102025-01-15T02:37:10.896ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70112025-01-15T02:37:10.896ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "195f9f65-e4ab-434c-b717-9af41e3f4796", "content-length": "372", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
70122025-01-15T02:37:10.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70132025-01-15T02:37:10.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70142025-01-15T02:37:10.908ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70152025-01-15T02:37:10.908ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 404.616959ms iteration = 21 status = {}
70162025-01-15T02:37:10.908ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70172025-01-15T02:37:10.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70182025-01-15T02:37:10.910ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70192025-01-15T02:37:10.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
70202025-01-15T02:37:10.911ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70212025-01-15T02:37:10.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
70222025-01-15T02:37:10.912ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70232025-01-15T02:37:10.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70242025-01-15T02:37:10.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
70252025-01-15T02:37:10.913ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70262025-01-15T02:37:10.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70272025-01-15T02:37:10.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175038 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a24e53f-b779-4b28-b17c-8fdf6aeda6bf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
70282025-01-15T02:37:10.916ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 412.718606ms iteration = 21 status = null
70292025-01-15T02:37:10.918ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
70302025-01-15T02:37:10.918ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "06aae4e0-994b-41ce-a81f-9a42e4975f68", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70312025-01-15T02:37:10.918ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70322025-01-15T02:37:10.921ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e4298eb0-5cc5-4961-972d-52b9efad01a6", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70332025-01-15T02:37:10.921ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70342025-01-15T02:37:10.922ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5f4ece26-f079-4761-9aff-bf675d5f1de5", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70352025-01-15T02:37:10.922ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70362025-01-15T02:37:10.925ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
70372025-01-15T02:37:10.925ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 421.885724ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
70382025-01-15T02:37:10.934ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
70392025-01-15T02:37:10.934ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 430.075605ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
70402025-01-15T02:37:10.938ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
70412025-01-15T02:37:10.938ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 434.099284ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
70422025-01-15T02:37:10.939ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70432025-01-15T02:37:10.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
70442025-01-15T02:37:10.942ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70452025-01-15T02:37:10.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70462025-01-15T02:37:10.943ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70472025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
70482025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70492025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
70502025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
70512025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70522025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70532025-01-15T02:37:10.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70542025-01-15T02:37:10.947ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
70552025-01-15T02:37:10.947ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 444.259798ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
70562025-01-15T02:37:10.948ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70572025-01-15T02:37:10.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70582025-01-15T02:37:10.948ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 444.479174ms iteration = 21 status = 0
70592025-01-15T02:37:10.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70602025-01-15T02:37:10.949ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70612025-01-15T02:37:10.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
70622025-01-15T02:37:10.951ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70632025-01-15T02:37:10.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70642025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
70652025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
70662025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
70672025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 449.479799ms iteration = 11 status = {"generation":2}
70682025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
70692025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70702025-01-15T02:37:10.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70712025-01-15T02:37:10.955ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70722025-01-15T02:37:10.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70732025-01-15T02:37:10.957ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
70742025-01-15T02:37:10.957ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
70752025-01-15T02:37:10.957ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70762025-01-15T02:37:10.957ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "aedb1d0c-6e20-4bb6-8020-439a2d038ffc", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
70772025-01-15T02:37:10.957ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70782025-01-15T02:37:10.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70792025-01-15T02:37:10.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70802025-01-15T02:37:10.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70812025-01-15T02:37:10.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70822025-01-15T02:37:10.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70832025-01-15T02:37:10.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70842025-01-15T02:37:10.966ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70852025-01-15T02:37:10.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70862025-01-15T02:37:10.968ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70872025-01-15T02:37:10.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70882025-01-15T02:37:10.969ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70892025-01-15T02:37:10.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70902025-01-15T02:37:10.970ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70912025-01-15T02:37:10.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
70922025-01-15T02:37:10.972ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70932025-01-15T02:37:10.972ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
70942025-01-15T02:37:10.972ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70952025-01-15T02:37:10.973ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ba5b465b-b917-4561-8edc-6a8cd170339f", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
70962025-01-15T02:37:10.974ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 470.63981ms iteration = 21 status = {}
70972025-01-15T02:37:10.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
70982025-01-15T02:37:10.975ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
70992025-01-15T02:37:10.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71002025-01-15T02:37:10.977ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71012025-01-15T02:37:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71022025-01-15T02:37:10.977ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
71032025-01-15T02:37:10.977ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
71042025-01-15T02:37:10.977ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71052025-01-15T02:37:10.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71062025-01-15T02:37:10.979ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71072025-01-15T02:37:10.979ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71082025-01-15T02:37:10.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71092025-01-15T02:37:10.981ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71102025-01-15T02:37:10.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71112025-01-15T02:37:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71122025-01-15T02:37:10.982ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71132025-01-15T02:37:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71142025-01-15T02:37:10.983ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71152025-01-15T02:37:10.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71162025-01-15T02:37:10.984ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
71172025-01-15T02:37:10.984ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
71182025-01-15T02:37:10.984ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
71192025-01-15T02:37:10.984ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 481.050231ms iteration = 11 status = {"generation":1}
71202025-01-15T02:37:10.985ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71212025-01-15T02:37:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71222025-01-15T02:37:10.985ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71232025-01-15T02:37:10.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71242025-01-15T02:37:10.986ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71252025-01-15T02:37:10.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71262025-01-15T02:37:10.988ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71272025-01-15T02:37:10.988ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
71282025-01-15T02:37:10.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71292025-01-15T02:37:10.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71302025-01-15T02:37:10.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
71312025-01-15T02:37:10.991ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71322025-01-15T02:37:10.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71332025-01-15T02:37:10.992ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71342025-01-15T02:37:10.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71352025-01-15T02:37:10.994ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71362025-01-15T02:37:10.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71372025-01-15T02:37:10.995ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71382025-01-15T02:37:10.998ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
71392025-01-15T02:37:10.998ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "719af2dc-c894-4559-864b-4e6e5d364283", "content-length": "134", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
71402025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
71412025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
71422025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
71432025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
71442025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71452025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71462025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71472025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
71482025-01-15T02:37:10.998ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
71492025-01-15T02:37:10.998ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71502025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
71512025-01-15T02:37:10.998ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
71522025-01-15T02:37:10.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71532025-01-15T02:37:10.999ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
71542025-01-15T02:37:10.999ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71552025-01-15T02:37:10.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71562025-01-15T02:37:11.000ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
71572025-01-15T02:37:11.000ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a00a852-fda9-425d-a207-09e88c3ad7db", "content-length": "2", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
71582025-01-15T02:37:11.000ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
71592025-01-15T02:37:11.000ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cd5ca63-cb74-44fe-b5e2-080a836b0a23", "content-length": "52", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
71602025-01-15T02:37:11.000ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
71612025-01-15T02:37:11.000ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
71622025-01-15T02:37:11.003ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71632025-01-15T02:37:11.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71642025-01-15T02:37:11.004ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71652025-01-15T02:37:11.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71662025-01-15T02:37:11.006ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71672025-01-15T02:37:11.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71682025-01-15T02:37:11.007ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71692025-01-15T02:37:11.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
71702025-01-15T02:37:11.008ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71712025-01-15T02:37:11.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71722025-01-15T02:37:11.009ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71732025-01-15T02:37:11.009ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71742025-01-15T02:37:11.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71752025-01-15T02:37:11.010ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
71762025-01-15T02:37:11.010ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71772025-01-15T02:37:11.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
71782025-01-15T02:37:11.011ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71792025-01-15T02:37:11.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71802025-01-15T02:37:11.012ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71812025-01-15T02:37:11.012ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71822025-01-15T02:37:11.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71832025-01-15T02:37:11.013ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71842025-01-15T02:37:11.014ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
71852025-01-15T02:37:11.014ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71862025-01-15T02:37:11.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
71872025-01-15T02:37:11.017ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
71882025-01-15T02:37:11.017ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71892025-01-15T02:37:11.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71902025-01-15T02:37:11.019ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71912025-01-15T02:37:11.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71922025-01-15T02:37:11.020ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71932025-01-15T02:37:11.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71942025-01-15T02:37:11.021ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71952025-01-15T02:37:11.025ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
71962025-01-15T02:37:11.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
71972025-01-15T02:37:11.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
71982025-01-15T02:37:11.030ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: b15378a9-5120-477b-abef-a170753103c1, kind: Service { id: 7468547b-b2f3-4993-afd8-72e9164a4301 }, name: Name("external-dns-7468547b-b2f3-4993-afd8-72e9164a4301"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 124])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: beea2870-fa6b-4d58-b0eb-af5a33149d11, kind: Service { id: 8b302e6b-b09c-4ef2-8768-1e5fae50d31a }, name: Name("external-dns-8b302e6b-b09c-4ef2-8768-1e5fae50d31a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 22195cd0-81ca-4c14-8294-e0bb8767fa51, kind: Service { id: 743f2600-b4a7-4963-8bef-692428b6800a }, name: Name("nexus-743f2600-b4a7-4963-8bef-692428b6800a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 161])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 357305ec-a785-4a6e-96cf-879dc3168076, kind: Service { id: a1e38c68-2155-4ce9-b70a-66d434b3f24f }, name: Name("nexus-a1e38c68-2155-4ce9-b70a-66d434b3f24f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 56f43fd0-1c39-449d-9089-5f10b7acf59b, kind: Service { id: b4bce764-527e-4a0d-9efa-46f5760fbb1d }, name: Name("nexus-b4bce764-527e-4a0d-9efa-46f5760fbb1d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 31])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
71992025-01-15T02:37:11.030ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72002025-01-15T02:37:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72012025-01-15T02:37:11.035ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72022025-01-15T02:37:11.037ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
72032025-01-15T02:37:11.037ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72042025-01-15T02:37:11.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72052025-01-15T02:37:11.039ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
72062025-01-15T02:37:11.039ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: b15378a9-5120-477b-abef-a170753103c1, kind: Service { id: 7468547b-b2f3-4993-afd8-72e9164a4301 }, name: Name("external-dns-7468547b-b2f3-4993-afd8-72e9164a4301"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 124])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: beea2870-fa6b-4d58-b0eb-af5a33149d11, kind: Service { id: 8b302e6b-b09c-4ef2-8768-1e5fae50d31a }, name: Name("external-dns-8b302e6b-b09c-4ef2-8768-1e5fae50d31a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 22195cd0-81ca-4c14-8294-e0bb8767fa51, kind: Service { id: 743f2600-b4a7-4963-8bef-692428b6800a }, name: Name("nexus-743f2600-b4a7-4963-8bef-692428b6800a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 161])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 357305ec-a785-4a6e-96cf-879dc3168076, kind: Service { id: a1e38c68-2155-4ce9-b70a-66d434b3f24f }, name: Name("nexus-a1e38c68-2155-4ce9-b70a-66d434b3f24f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 56f43fd0-1c39-449d-9089-5f10b7acf59b, kind: Service { id: b4bce764-527e-4a0d-9efa-46f5760fbb1d }, name: Name("nexus-b4bce764-527e-4a0d-9efa-46f5760fbb1d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 31])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
72072025-01-15T02:37:11.039ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
72082025-01-15T02:37:11.091ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: ce66c06a-7d54-47a6-860d-59da68080fee, time_created: 2025-01-15T02:27:10.189884Z, time_modified: 2025-01-15T02:27:10.189884Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
72092025-01-15T02:37:11.091ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
72102025-01-15T02:37:11.091ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72112025-01-15T02:37:11.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72122025-01-15T02:37:11.094ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
72132025-01-15T02:37:11.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
72142025-01-15T02:37:11.112ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
72152025-01-15T02:37:11.114ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "df755040-e2c9-4792-8a96-90211ab6ebbc", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
72162025-01-15T02:37:11.114ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
72172025-01-15T02:37:11.114ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
72182025-01-15T02:37:11.115ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "43611c32-0a9b-4809-b74e-4c2ce4170514", "content-length": "682", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
72192025-01-15T02:37:11.115ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72202025-01-15T02:37:11.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72212025-01-15T02:37:11.115ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "639bafdb-afc4-44f1-b354-0df557e93c73", "content-length": "691", "date": "Wed, 15 Jan 2025 02:37:10 GMT"} })
72222025-01-15T02:37:11.115ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
72232025-01-15T02:37:11.115ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
72242025-01-15T02:37:11.115ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
72252025-01-15T02:37:11.117ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 613.053673ms iteration = 21 status = {}
72262025-01-15T02:37:11.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72272025-01-15T02:37:11.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72282025-01-15T02:37:11.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72292025-01-15T02:37:11.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72302025-01-15T02:37:11.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72312025-01-15T02:37:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72322025-01-15T02:37:11.133ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 629.342412ms iteration = 21 status = {}
72332025-01-15T02:37:11.141ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72342025-01-15T02:37:11.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
72352025-01-15T02:37:11.173ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
72362025-01-15T02:37:11.175ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "bfff024d-cb3c-47e0-9307-48144f751d77", "date": "Wed, 15 Jan 2025 02:37:11 GMT"} })
72372025-01-15T02:37:11.177ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
72382025-01-15T02:37:11.177ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 673.098092ms
72392025-01-15T02:37:11.177ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 673.109623ms iteration = 3 status = {}
72402025-01-15T02:37:11.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfdfdd55-e44c-430e-819d-ea7b8151fed7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72412025-01-15T02:37:11.940ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfdfdd55-e44c-430e-819d-ea7b8151fed7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72422025-01-15T02:37:11.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2387 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dfdfdd55-e44c-430e-819d-ea7b8151fed7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72432025-01-15T02:37:12.960ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 53b96e9e-93cd-4d4d-9303-07ae5c3aff6e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72442025-01-15T02:37:12.961ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 53b96e9e-93cd-4d4d-9303-07ae5c3aff6e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72452025-01-15T02:37:12.962ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1748 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 53b96e9e-93cd-4d4d-9303-07ae5c3aff6e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72462025-01-15T02:37:13.764ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
72472025-01-15T02:37:13.764ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72482025-01-15T02:37:13.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72492025-01-15T02:37:13.767ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502025-01-15T02:37:13.768ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
72512025-01-15T02:37:13.768ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72522025-01-15T02:37:13.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72532025-01-15T02:37:13.777ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(02aa3eb6-cd94-47ee-800c-98a606109c2c, 0), (b9ab1151-0703-4633-bd71-1789064f9f67, 0), (a3fbae06-8eed-4f0e-807b-df7a6415b8d3, 0), (a859ac93-c30e-4bf4-abd5-125ffc8eedb0, 0)] collection_id = 02aa3eb6-cd94-47ee-800c-98a606109c2c
72542025-01-15T02:37:13.777ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72552025-01-15T02:37:13.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72562025-01-15T02:37:13.880ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 02aa3eb6-cd94-47ee-800c-98a606109c2c file = nexus/db-queries/src/db/datastore/inventory.rs:1442 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
72572025-01-15T02:37:13.880ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72582025-01-15T02:37:13.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72592025-01-15T02:37:13.890ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(b9ab1151-0703-4633-bd71-1789064f9f67, 0), (a3fbae06-8eed-4f0e-807b-df7a6415b8d3, 0), (a859ac93-c30e-4bf4-abd5-125ffc8eedb0, 0), (c8892e41-3aa5-4991-8cb6-b253894c1a4c, 0)] collection_id = b9ab1151-0703-4633-bd71-1789064f9f67
72602025-01-15T02:37:13.890ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72612025-01-15T02:37:13.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72622025-01-15T02:37:13.963ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = b9ab1151-0703-4633-bd71-1789064f9f67 file = nexus/db-queries/src/db/datastore/inventory.rs:1442 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
72632025-01-15T02:37:13.963ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72642025-01-15T02:37:13.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72652025-01-15T02:37:13.969ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(a3fbae06-8eed-4f0e-807b-df7a6415b8d3, 0), (a859ac93-c30e-4bf4-abd5-125ffc8eedb0, 0), (c8892e41-3aa5-4991-8cb6-b253894c1a4c, 0)]
72662025-01-15T02:37:13.970ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3920, tv_nsec: 807663481 } })
72672025-01-15T02:37:14.000ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a4a5ffd-eace-452d-bbd6-0988a8af6fb0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72682025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
72692025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
72702025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
72712025-01-15T02:37:14.001ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a4a5ffd-eace-452d-bbd6-0988a8af6fb0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72722025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3852c1fb-2316-4889-9b46-08f57b9d65c6", "content-length": "476", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72732025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
72742025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "732312a5-8107-4d17-88f1-381551a5bf3d", "content-length": "734", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72752025-01-15T02:37:14.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
72762025-01-15T02:37:14.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8815f0d6-0467-4c30-9150-de4db28dae25", "content-length": "111", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72772025-01-15T02:37:14.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
72782025-01-15T02:37:14.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46178f0e-ada8-4b68-9cca-3af94a4e4204", "content-length": "111", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72792025-01-15T02:37:14.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
72802025-01-15T02:37:14.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8670fa0-ce0d-4a58-896f-35581e48b92d", "content-length": "167", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72812025-01-15T02:37:14.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
72822025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f335c23c-c628-4b79-bbb6-e0de92cd9ecf", "content-length": "167", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72832025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
72842025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe4f8a2-d4b0-4b15-9f07-df8eec788831", "content-length": "175", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72852025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
72862025-01-15T02:37:14.003ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2698 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a4a5ffd-eace-452d-bbd6-0988a8af6fb0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
72872025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8c88766-e92d-48da-8aab-61005bf06df6", "content-length": "175", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72882025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
72892025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ceec4d2-252c-4019-805c-f57a6cb34e90", "content-length": "702", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72902025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
72912025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b3c9c3e-f754-4de2-bae2-3b97455839b2", "content-length": "727", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72922025-01-15T02:37:14.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
72932025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fcbd05d-4fe6-463a-b6b6-498ab003c4be", "content-length": "729", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72942025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
72952025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ab04740-d56a-42bc-84b6-c9ba47785092", "content-length": "728", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72962025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
72972025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8750669f-99c8-475f-b949-beba17ba636e", "content-length": "722", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
72982025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
72992025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca6324f5-1678-49da-9212-e487d1eae5a2", "content-length": "109", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73002025-01-15T02:37:14.004ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
73012025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b5f84f5-a980-4e52-85e1-8ff3a8452aff", "content-length": "109", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73022025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
73032025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24576e5f-f3ab-4847-b679-c72fb935a55d", "content-length": "169", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73042025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
73052025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e208642-8806-41eb-8a73-05da8f70ba60", "content-length": "169", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73062025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
73072025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae004b72-5e44-476e-abc7-3ea7d64a564f", "content-length": "178", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73082025-01-15T02:37:14.005ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
73092025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "347e8631-99c8-48e1-8903-2981da54a307", "content-length": "178", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73102025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
73112025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "068fb3ff-352a-4fe0-ab8d-46df53f96f91", "content-length": "702", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73122025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
73132025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fed50712-7dcf-41e5-bee8-7e3a6c98e615", "content-length": "727", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73142025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
73152025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "379f0a05-45d8-422f-adb1-88378675ab9a", "content-length": "729", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73162025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
73172025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "39543d46-04fc-4355-9ec8-ecd9000885fc", "content-length": "728", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73182025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73192025-01-15T02:37:14.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73202025-01-15T02:37:14.008ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73212025-01-15T02:37:14.010ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
73222025-01-15T02:37:14.010ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73232025-01-15T02:37:14.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73242025-01-15T02:37:14.045ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73252025-01-15T02:37:14.047ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
73262025-01-15T02:37:14.047ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73272025-01-15T02:37:14.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73282025-01-15T02:37:14.080ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
73292025-01-15T02:37:14.081ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
73302025-01-15T02:37:14.287ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "67660328-5733-495b-ab43-f54b0c4efa75", "content-length": "30799", "date": "Wed, 15 Jan 2025 02:37:14 GMT"} })
73312025-01-15T02:37:14.289ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
73322025-01-15T02:37:14.289ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
73332025-01-15T02:37:14.289ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
73342025-01-15T02:37:14.291ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73352025-01-15T02:37:14.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73362025-01-15T02:37:14.293ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73372025-01-15T02:37:14.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
73382025-01-15T02:37:14.295ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73392025-01-15T02:37:14.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73402025-01-15T02:37:14.455ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = be8b9b61-5b70-46b9-991d-cdd88103127c file = nexus/db-queries/src/db/datastore/inventory.rs:1010
73412025-01-15T02:37:14.455ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = be8b9b61-5b70-46b9-991d-cdd88103127c time_started = 2025-01-15 02:37:14.001125 UTC
73422025-01-15T02:37:14.455ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 690.696967ms iteration = 3 status = {"collection_id":"be8b9b61-5b70-46b9-991d-cdd88103127c","time_done":"2025-01-15 02:37:14.289693 UTC","time_started":"2025-01-15 02:37:14.001125 UTC"}
73432025-01-15T02:37:14.455ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
73442025-01-15T02:37:14.455ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73452025-01-15T02:37:14.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73462025-01-15T02:37:14.456ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73472025-01-15T02:37:14.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
73482025-01-15T02:37:14.458ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73492025-01-15T02:37:14.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73502025-01-15T02:37:14.460ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73512025-01-15T02:37:14.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73522025-01-15T02:37:14.462ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73532025-01-15T02:37:14.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
73542025-01-15T02:37:14.463ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73552025-01-15T02:37:14.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73562025-01-15T02:37:14.502ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 46.620318ms iteration = 24 status = {"physical_disks_added":0}
73572025-01-15T02:37:15.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0a2fd412-e4e5-416a-b07f-2dc98d8bd3fa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73582025-01-15T02:37:15.029ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0a2fd412-e4e5-416a-b07f-2dc98d8bd3fa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73592025-01-15T02:37:15.031ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0a2fd412-e4e5-416a-b07f-2dc98d8bd3fa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73602025-01-15T02:37:15.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 496 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = aa6b78cc-0e01-49af-9642-a3e0ecb69170 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
73612025-01-15T02:37:16.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cf24bfc-6532-44c8-b656-3822aa17033e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73622025-01-15T02:37:16.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cf24bfc-6532-44c8-b656-3822aa17033e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73632025-01-15T02:37:16.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2586 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2cf24bfc-6532-44c8-b656-3822aa17033e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73642025-01-15T02:37:17.075ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23bd788c-cea3-4012-a323-f619019404cf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73652025-01-15T02:37:17.075ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23bd788c-cea3-4012-a323-f619019404cf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73662025-01-15T02:37:17.077ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2617 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 23bd788c-cea3-4012-a323-f619019404cf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73672025-01-15T02:37:18.097ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cceaa5f1-1604-4b20-88a5-1856ac910c94 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73682025-01-15T02:37:18.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cceaa5f1-1604-4b20-88a5-1856ac910c94 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73692025-01-15T02:37:18.099ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2217 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cceaa5f1-1604-4b20-88a5-1856ac910c94 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73702025-01-15T02:37:19.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ad9ef96b-4a7e-42c0-af70-7cf75d226b13 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73712025-01-15T02:37:19.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ad9ef96b-4a7e-42c0-af70-7cf75d226b13 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73722025-01-15T02:37:19.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2589 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ad9ef96b-4a7e-42c0-af70-7cf75d226b13 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73732025-01-15T02:37:20.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a54d24b-445c-4d06-9e9f-71cb4940a647 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73742025-01-15T02:37:20.143ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a54d24b-445c-4d06-9e9f-71cb4940a647 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73752025-01-15T02:37:20.145ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2548 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9a54d24b-445c-4d06-9e9f-71cb4940a647 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73762025-01-15T02:37:20.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
73772025-01-15T02:37:20.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
73782025-01-15T02:37:20.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73792025-01-15T02:37:20.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73802025-01-15T02:37:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73812025-01-15T02:37:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73822025-01-15T02:37:20.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73832025-01-15T02:37:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73842025-01-15T02:37:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73852025-01-15T02:37:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73862025-01-15T02:37:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73872025-01-15T02:37:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73882025-01-15T02:37:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73892025-01-15T02:37:20.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
73902025-01-15T02:37:20.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73912025-01-15T02:37:20.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73922025-01-15T02:37:20.529ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
73932025-01-15T02:37:20.529ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.597711ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
73942025-01-15T02:37:20.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73952025-01-15T02:37:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73962025-01-15T02:37:20.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.461135ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73972025-01-15T02:37:21.165ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 716dee89-7c71-47e4-88c9-391adec9d523 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73982025-01-15T02:37:21.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 716dee89-7c71-47e4-88c9-391adec9d523 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
73992025-01-15T02:37:21.168ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2690 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 716dee89-7c71-47e4-88c9-391adec9d523 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74002025-01-15T02:37:22.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e9ccbd64-463b-4dd0-89ff-b0beb681646d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74012025-01-15T02:37:22.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e9ccbd64-463b-4dd0-89ff-b0beb681646d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74022025-01-15T02:37:22.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e9ccbd64-463b-4dd0-89ff-b0beb681646d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74032025-01-15T02:37:23.209ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c741ec8c-5dc9-427a-8667-8b62b5c60212 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74042025-01-15T02:37:23.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c741ec8c-5dc9-427a-8667-8b62b5c60212 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74052025-01-15T02:37:23.212ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2379 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c741ec8c-5dc9-427a-8667-8b62b5c60212 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74062025-01-15T02:37:24.232ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4dd181ad-33d3-4868-b8cc-bf42d756b5b9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74072025-01-15T02:37:24.233ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4dd181ad-33d3-4868-b8cc-bf42d756b5b9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74082025-01-15T02:37:24.235ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2396 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4dd181ad-33d3-4868-b8cc-bf42d756b5b9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74092025-01-15T02:37:25.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a82d5f6-7396-4e22-8618-8e178646ce6d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74102025-01-15T02:37:25.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a82d5f6-7396-4e22-8618-8e178646ce6d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74112025-01-15T02:37:25.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2022 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6a82d5f6-7396-4e22-8618-8e178646ce6d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74122025-01-15T02:37:25.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 400 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 5f887204-d8e8-42ed-9822-bf2e1bf792ec response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
74132025-01-15T02:37:26.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 582621d3-17c0-459e-aa6e-1f343cbb9eff roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74142025-01-15T02:37:26.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 582621d3-17c0-459e-aa6e-1f343cbb9eff resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74152025-01-15T02:37:26.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2413 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 582621d3-17c0-459e-aa6e-1f343cbb9eff response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74162025-01-15T02:37:27.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 12da5bad-8e9e-4a4e-a294-541774dfb12e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74172025-01-15T02:37:27.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 12da5bad-8e9e-4a4e-a294-541774dfb12e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74182025-01-15T02:37:27.300ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2047 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 12da5bad-8e9e-4a4e-a294-541774dfb12e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74192025-01-15T02:37:28.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28364e6b-a325-4575-91f4-925395d7880c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74202025-01-15T02:37:28.321ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28364e6b-a325-4575-91f4-925395d7880c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74212025-01-15T02:37:28.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1938 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28364e6b-a325-4575-91f4-925395d7880c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74222025-01-15T02:37:29.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e64db38-0fae-4794-9996-c52d26a89876 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74232025-01-15T02:37:29.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e64db38-0fae-4794-9996-c52d26a89876 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74242025-01-15T02:37:29.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2410 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e64db38-0fae-4794-9996-c52d26a89876 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74252025-01-15T02:37:30.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a572bf60-2b01-4a03-837b-14fc66404eed roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74262025-01-15T02:37:30.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a572bf60-2b01-4a03-837b-14fc66404eed resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74272025-01-15T02:37:30.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2346 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a572bf60-2b01-4a03-837b-14fc66404eed response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74282025-01-15T02:37:30.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
74292025-01-15T02:37:30.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
74302025-01-15T02:37:30.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74312025-01-15T02:37:30.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74322025-01-15T02:37:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74332025-01-15T02:37:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74342025-01-15T02:37:30.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74352025-01-15T02:37:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74362025-01-15T02:37:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74372025-01-15T02:37:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74382025-01-15T02:37:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74392025-01-15T02:37:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74402025-01-15T02:37:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74412025-01-15T02:37:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74422025-01-15T02:37:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74432025-01-15T02:37:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
74442025-01-15T02:37:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74452025-01-15T02:37:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74462025-01-15T02:37:30.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
74472025-01-15T02:37:30.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.945322ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
74482025-01-15T02:37:30.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.648097ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74492025-01-15T02:37:30.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = a59c07df-e8ed-401b-86b5-006ecf308bcb roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
74502025-01-15T02:37:30.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = a59c07df-e8ed-401b-86b5-006ecf308bcb resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
74512025-01-15T02:37:30.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = a59c07df-e8ed-401b-86b5-006ecf308bcb response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
74522025-01-15T02:37:30.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 605cc201-369c-411a-8967-0db3a8a0f8d8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
74532025-01-15T02:37:30.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 605cc201-369c-411a-8967-0db3a8a0f8d8 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
74542025-01-15T02:37:30.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2397 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 605cc201-369c-411a-8967-0db3a8a0f8d8 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
74552025-01-15T02:37:31.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3afeca04-df39-4140-9c77-3d6159707949 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74562025-01-15T02:37:31.389ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3afeca04-df39-4140-9c77-3d6159707949 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74572025-01-15T02:37:31.391ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2130 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3afeca04-df39-4140-9c77-3d6159707949 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74582025-01-15T02:37:32.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c2914c08-4ca8-40d4-b8a0-df171d0e8ab7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74592025-01-15T02:37:32.412ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c2914c08-4ca8-40d4-b8a0-df171d0e8ab7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74602025-01-15T02:37:32.414ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2065 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c2914c08-4ca8-40d4-b8a0-df171d0e8ab7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74612025-01-15T02:37:33.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b3189e4-a518-4cfa-a78e-41405c8195f9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74622025-01-15T02:37:33.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b3189e4-a518-4cfa-a78e-41405c8195f9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74632025-01-15T02:37:33.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2695 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b3189e4-a518-4cfa-a78e-41405c8195f9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74642025-01-15T02:37:34.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a14883ba-3d93-45ec-b80b-ca718e86eff8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74652025-01-15T02:37:34.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a14883ba-3d93-45ec-b80b-ca718e86eff8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74662025-01-15T02:37:34.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1560 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a14883ba-3d93-45ec-b80b-ca718e86eff8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74672025-01-15T02:37:35.479ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 842140bb-b634-4dde-9cfd-147607155ae8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74682025-01-15T02:37:35.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 842140bb-b634-4dde-9cfd-147607155ae8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74692025-01-15T02:37:35.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2042 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 842140bb-b634-4dde-9cfd-147607155ae8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74702025-01-15T02:37:35.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 444 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = b053f862-ff29-4143-b8a6-b23a1ea717b2 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
74712025-01-15T02:37:36.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = abc0596f-a9bb-4226-b1ed-2ed7ea6e939e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74722025-01-15T02:37:36.503ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = abc0596f-a9bb-4226-b1ed-2ed7ea6e939e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74732025-01-15T02:37:36.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1993 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = abc0596f-a9bb-4226-b1ed-2ed7ea6e939e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74742025-01-15T02:37:37.525ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2596756f-67b1-4439-a509-36a349eb52ea roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74752025-01-15T02:37:37.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2596756f-67b1-4439-a509-36a349eb52ea resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74762025-01-15T02:37:37.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2635 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2596756f-67b1-4439-a509-36a349eb52ea response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74772025-01-15T02:37:38.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58069b09-776e-4d1b-af8e-7fe69db5e788 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74782025-01-15T02:37:38.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58069b09-776e-4d1b-af8e-7fe69db5e788 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74792025-01-15T02:37:38.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2145 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 58069b09-776e-4d1b-af8e-7fe69db5e788 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74802025-01-15T02:37:39.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4222fe4-954c-4236-8cc1-150418c4f4f8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74812025-01-15T02:37:39.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4222fe4-954c-4236-8cc1-150418c4f4f8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74822025-01-15T02:37:39.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1665 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4222fe4-954c-4236-8cc1-150418c4f4f8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
74832025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
74842025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
74852025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
74862025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
74872025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
74882025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
74892025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
74902025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
74912025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
74922025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74932025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
74942025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
74952025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74962025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
74972025-01-15T02:37:40.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
74982025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74992025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
75002025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
75012025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75022025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75032025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
75042025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
75052025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
75062025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75072025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
75082025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
75092025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75102025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
75112025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
75122025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
75132025-01-15T02:37:40.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75142025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
75152025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75162025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75172025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75182025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75192025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75202025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75212025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75222025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75232025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
75242025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
75252025-01-15T02:37:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
75262025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
75272025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
75282025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
75292025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
75302025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
75312025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
75322025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
75332025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
75342025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
75352025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
75362025-01-15T02:37:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
75372025-01-15T02:37:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3947, tv_nsec: 344170542 } })
75382025-01-15T02:37:40.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
75392025-01-15T02:37:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3947, tv_nsec: 344646724 } })
75402025-01-15T02:37:40.507ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.784682ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
75412025-01-15T02:37:40.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
75422025-01-15T02:37:40.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75432025-01-15T02:37:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75442025-01-15T02:37:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75452025-01-15T02:37:40.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75462025-01-15T02:37:40.508ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75472025-01-15T02:37:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
75482025-01-15T02:37:40.509ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
75492025-01-15T02:37:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
75502025-01-15T02:37:40.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75512025-01-15T02:37:40.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75522025-01-15T02:37:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75532025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
75542025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75552025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75562025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75572025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75582025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75592025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
75602025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75612025-01-15T02:37:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75622025-01-15T02:37:40.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75632025-01-15T02:37:40.513ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75642025-01-15T02:37:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75652025-01-15T02:37:40.515ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75662025-01-15T02:37:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75672025-01-15T02:37:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
75682025-01-15T02:37:40.515ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75692025-01-15T02:37:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75702025-01-15T02:37:40.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75712025-01-15T02:37:40.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75722025-01-15T02:37:40.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75732025-01-15T02:37:40.523ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75742025-01-15T02:37:40.523ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75752025-01-15T02:37:40.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3947, tv_nsec: 361080238 } })
75762025-01-15T02:37:40.523ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
75772025-01-15T02:37:40.523ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
75782025-01-15T02:37:40.523ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
75792025-01-15T02:37:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75802025-01-15T02:37:40.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
75812025-01-15T02:37:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
75822025-01-15T02:37:40.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75832025-01-15T02:37:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75842025-01-15T02:37:40.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75852025-01-15T02:37:40.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75862025-01-15T02:37:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75872025-01-15T02:37:40.527ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
75882025-01-15T02:37:40.527ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
75892025-01-15T02:37:40.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
75902025-01-15T02:37:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
75912025-01-15T02:37:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
75922025-01-15T02:37:40.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75932025-01-15T02:37:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75942025-01-15T02:37:40.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75952025-01-15T02:37:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75962025-01-15T02:37:40.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75972025-01-15T02:37:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75982025-01-15T02:37:40.550ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
75992025-01-15T02:37:40.550ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.06116ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
76002025-01-15T02:37:40.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76012025-01-15T02:37:40.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76022025-01-15T02:37:40.556ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.457928ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
76032025-01-15T02:37:40.558ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76042025-01-15T02:37:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
76052025-01-15T02:37:40.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76062025-01-15T02:37:40.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76072025-01-15T02:37:40.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 57.42169ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
76082025-01-15T02:37:40.562ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76092025-01-15T02:37:40.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76102025-01-15T02:37:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76112025-01-15T02:37:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76122025-01-15T02:37:40.563ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
76132025-01-15T02:37:40.563ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 60.103152ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
76142025-01-15T02:37:40.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76152025-01-15T02:37:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76162025-01-15T02:37:40.566ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76172025-01-15T02:37:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
76182025-01-15T02:37:40.569ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76192025-01-15T02:37:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76202025-01-15T02:37:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 66.948876ms iteration = 22 status = 0
76212025-01-15T02:37:40.573ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76222025-01-15T02:37:40.573ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
76232025-01-15T02:37:40.573ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 69.846495ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
76242025-01-15T02:37:40.575ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76252025-01-15T02:37:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76262025-01-15T02:37:40.577ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76272025-01-15T02:37:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76282025-01-15T02:37:40.579ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76292025-01-15T02:37:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76302025-01-15T02:37:40.589ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.790844ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76312025-01-15T02:37:40.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb8a1294-9967-4a41-9c7d-eea94f793d52 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
76322025-01-15T02:37:40.593ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb8a1294-9967-4a41-9c7d-eea94f793d52 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
76332025-01-15T02:37:40.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2195 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bb8a1294-9967-4a41-9c7d-eea94f793d52 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
76342025-01-15T02:37:40.601ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
76352025-01-15T02:37:40.601ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76362025-01-15T02:37:40.601ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c52133c-5c29-4d29-8353-e61729d2a1ae", "content-length": "26", "date": "Wed, 15 Jan 2025 02:37:39 GMT"} })
76372025-01-15T02:37:40.601ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76382025-01-15T02:37:40.603ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
76392025-01-15T02:37:40.609ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 106.037842ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76402025-01-15T02:37:40.610ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.674801ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76412025-01-15T02:37:40.611ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76422025-01-15T02:37:40.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76432025-01-15T02:37:40.626ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
76442025-01-15T02:37:40.626ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 122.583528ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
76452025-01-15T02:37:40.631ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76462025-01-15T02:37:40.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76472025-01-15T02:37:40.640ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76482025-01-15T02:37:40.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76492025-01-15T02:37:40.641ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76502025-01-15T02:37:40.643ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 138.485187ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
76512025-01-15T02:37:40.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
76522025-01-15T02:37:40.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76532025-01-15T02:37:40.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76542025-01-15T02:37:40.645ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76552025-01-15T02:37:40.646ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
76562025-01-15T02:37:40.646ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76572025-01-15T02:37:40.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76582025-01-15T02:37:40.655ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 151.639382ms iteration = 25 status = {"physical_disks_added":0}
76592025-01-15T02:37:40.690ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
76602025-01-15T02:37:40.690ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76612025-01-15T02:37:40.690ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ead1bb9c-d55c-4860-aa11-9e59b54e9b9c", "content-length": "26", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
76622025-01-15T02:37:40.690ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76632025-01-15T02:37:40.693ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
76642025-01-15T02:37:40.725ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
76652025-01-15T02:37:40.725ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76662025-01-15T02:37:40.725ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31ff1b3b-a4df-4255-af35-aaf8e5623533", "content-length": "26", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
76672025-01-15T02:37:40.725ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76682025-01-15T02:37:40.728ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
76692025-01-15T02:37:40.742ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
76702025-01-15T02:37:40.742ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ff9c139-8724-41c4-b746-ca3a09d03878", "content-length": "1", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
76712025-01-15T02:37:40.744ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
76722025-01-15T02:37:40.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
76732025-01-15T02:37:40.751ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
76742025-01-15T02:37:40.751ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
76752025-01-15T02:37:40.753ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 250.686864ms iteration = 22 status = null
76762025-01-15T02:37:40.770ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
76772025-01-15T02:37:40.788ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 285.489064ms iteration = 22 status = {}
76782025-01-15T02:37:40.820ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ef3b2ed-5d7a-4906-96ae-e71ba71632ea", "content-length": "372", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
76792025-01-15T02:37:40.820ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76802025-01-15T02:37:40.820ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b4f25f2-7e36-4dae-b04c-058b26fc05b7", "content-length": "691", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
76812025-01-15T02:37:40.820ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
76822025-01-15T02:37:40.820ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
76832025-01-15T02:37:40.820ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
76842025-01-15T02:37:40.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76852025-01-15T02:37:40.821ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 317.983251ms iteration = 22 status = {}
76862025-01-15T02:37:40.867ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
76872025-01-15T02:37:40.867ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
76882025-01-15T02:37:40.867ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "78bb187c-7cb6-43c9-b526-caa75d9bc202", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
76892025-01-15T02:37:40.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76902025-01-15T02:37:40.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76912025-01-15T02:37:40.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76922025-01-15T02:37:40.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76932025-01-15T02:37:40.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76942025-01-15T02:37:40.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76952025-01-15T02:37:40.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76962025-01-15T02:37:40.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76972025-01-15T02:37:40.873ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
76982025-01-15T02:37:40.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76992025-01-15T02:37:40.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77002025-01-15T02:37:40.876ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
77012025-01-15T02:37:40.876ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77022025-01-15T02:37:40.877ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "04a4d52f-22b2-4e55-a27f-55e1db205d9c", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
77032025-01-15T02:37:40.879ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 375.260927ms iteration = 22 status = {}
77042025-01-15T02:37:40.882ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
77052025-01-15T02:37:40.882ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1183950c-9454-4346-a917-266c4b1a640f", "content-length": "134", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
77062025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
77072025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
77082025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
77092025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
77102025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77112025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77122025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77132025-01-15T02:37:40.882ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77142025-01-15T02:37:40.883ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77152025-01-15T02:37:40.883ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77162025-01-15T02:37:40.883ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
77172025-01-15T02:37:40.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77182025-01-15T02:37:40.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77192025-01-15T02:37:40.885ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
77202025-01-15T02:37:40.885ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "02532224-790c-4e13-87c1-e4ae9cba1e2b", "content-length": "2", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
77212025-01-15T02:37:40.885ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
77222025-01-15T02:37:40.886ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfe99dd5-14fd-42f5-abe1-8e67666d211e", "content-length": "52", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
77232025-01-15T02:37:40.886ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
77242025-01-15T02:37:40.886ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
77252025-01-15T02:37:40.986ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5e78797a-c492-4892-8983-944819938a0d", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
77262025-01-15T02:37:40.986ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
77272025-01-15T02:37:40.986ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
77282025-01-15T02:37:40.987ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "46ffffae-ba24-4dd1-bd8f-ebf52b688e33", "content-length": "682", "date": "Wed, 15 Jan 2025 02:37:40 GMT"} })
77292025-01-15T02:37:40.987ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77302025-01-15T02:37:40.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77312025-01-15T02:37:40.990ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77322025-01-15T02:37:40.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77332025-01-15T02:37:40.994ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77342025-01-15T02:37:40.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77352025-01-15T02:37:40.996ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77362025-01-15T02:37:40.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77372025-01-15T02:37:41.003ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 499.907459ms iteration = 22 status = {}
77382025-01-15T02:37:41.618ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d1942def-bc17-445b-80f6-1b912bd64f55 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77392025-01-15T02:37:41.619ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d1942def-bc17-445b-80f6-1b912bd64f55 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77402025-01-15T02:37:41.620ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1975 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d1942def-bc17-445b-80f6-1b912bd64f55 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77412025-01-15T02:37:42.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f70dea6f-6490-4c05-8808-0e8c358eea9b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77422025-01-15T02:37:42.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f70dea6f-6490-4c05-8808-0e8c358eea9b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77432025-01-15T02:37:42.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1917 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f70dea6f-6490-4c05-8808-0e8c358eea9b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77442025-01-15T02:37:43.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4b7e43c-b312-4595-b3a0-9c40ba333a71 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77452025-01-15T02:37:43.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4b7e43c-b312-4595-b3a0-9c40ba333a71 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77462025-01-15T02:37:43.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2325 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c4b7e43c-b312-4595-b3a0-9c40ba333a71 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77472025-01-15T02:37:44.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e1e2c00-ce18-4679-92bc-5caf93c22ae8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77482025-01-15T02:37:44.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e1e2c00-ce18-4679-92bc-5caf93c22ae8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77492025-01-15T02:37:44.686ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2170 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0e1e2c00-ce18-4679-92bc-5caf93c22ae8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77502025-01-15T02:37:45.705ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9d77e8c5-2017-4f91-aafb-64bf7304a967 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77512025-01-15T02:37:45.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9d77e8c5-2017-4f91-aafb-64bf7304a967 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77522025-01-15T02:37:45.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2239 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9d77e8c5-2017-4f91-aafb-64bf7304a967 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77532025-01-15T02:37:45.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 77e65725-4702-4bce-ad80-27eda9a5b06e roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
77542025-01-15T02:37:45.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 77e65725-4702-4bce-ad80-27eda9a5b06e resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
77552025-01-15T02:37:45.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2059 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 77e65725-4702-4bce-ad80-27eda9a5b06e response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
77562025-01-15T02:37:45.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 4a3a04ac-2e27-44bb-9c5a-89b9a6bfe63d roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
77572025-01-15T02:37:45.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 4a3a04ac-2e27-44bb-9c5a-89b9a6bfe63d resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
77582025-01-15T02:37:45.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1862 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = 4a3a04ac-2e27-44bb-9c5a-89b9a6bfe63d response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
77592025-01-15T02:37:45.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 398 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 864efe6a-ef2f-484b-8b56-6aa74f32e310 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
77602025-01-15T02:37:46.726ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b26aefe5-b910-4edf-acde-ed5c1f54c68d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77612025-01-15T02:37:46.727ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b26aefe5-b910-4edf-acde-ed5c1f54c68d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77622025-01-15T02:37:46.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2077 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b26aefe5-b910-4edf-acde-ed5c1f54c68d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77632025-01-15T02:37:47.749ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8310d194-281a-4e90-a65c-3bba4210e288 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77642025-01-15T02:37:47.749ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8310d194-281a-4e90-a65c-3bba4210e288 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77652025-01-15T02:37:47.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2458 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8310d194-281a-4e90-a65c-3bba4210e288 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77662025-01-15T02:37:48.775ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6421041d-ccd3-411b-95dc-b75f02973d43 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77672025-01-15T02:37:48.775ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6421041d-ccd3-411b-95dc-b75f02973d43 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77682025-01-15T02:37:48.777ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2522 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6421041d-ccd3-411b-95dc-b75f02973d43 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77692025-01-15T02:37:49.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a96a8db8-3d6e-4685-821a-5f9fdb85999b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77702025-01-15T02:37:49.799ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a96a8db8-3d6e-4685-821a-5f9fdb85999b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77712025-01-15T02:37:49.801ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2685 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a96a8db8-3d6e-4685-821a-5f9fdb85999b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77722025-01-15T02:37:50.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
77732025-01-15T02:37:50.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
77742025-01-15T02:37:50.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77752025-01-15T02:37:50.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77762025-01-15T02:37:50.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77772025-01-15T02:37:50.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77782025-01-15T02:37:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77792025-01-15T02:37:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
77802025-01-15T02:37:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77812025-01-15T02:37:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77822025-01-15T02:37:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77832025-01-15T02:37:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77842025-01-15T02:37:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77852025-01-15T02:37:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
77862025-01-15T02:37:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77872025-01-15T02:37:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77882025-01-15T02:37:50.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77892025-01-15T02:37:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77902025-01-15T02:37:50.535ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.314965ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77912025-01-15T02:37:50.574ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
77922025-01-15T02:37:50.574ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.908943ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
77932025-01-15T02:37:50.831ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 042cfb28-aa36-479d-8528-a71f17bc5f2a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77942025-01-15T02:37:50.831ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 042cfb28-aa36-479d-8528-a71f17bc5f2a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77952025-01-15T02:37:50.833ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2210 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 042cfb28-aa36-479d-8528-a71f17bc5f2a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77962025-01-15T02:37:51.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1872dc38-adf8-4b08-94aa-f77c7f7a7de9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77972025-01-15T02:37:51.856ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1872dc38-adf8-4b08-94aa-f77c7f7a7de9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77982025-01-15T02:37:51.858ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2364 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1872dc38-adf8-4b08-94aa-f77c7f7a7de9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
77992025-01-15T02:37:52.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa2a01ec-6e34-4fae-bac8-08da165321ea roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78002025-01-15T02:37:52.878ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa2a01ec-6e34-4fae-bac8-08da165321ea resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78012025-01-15T02:37:52.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2280 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = aa2a01ec-6e34-4fae-bac8-08da165321ea response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78022025-01-15T02:37:53.900ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2d5eaafc-9584-4c16-8174-9e1cd2cae488 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78032025-01-15T02:37:53.900ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2d5eaafc-9584-4c16-8174-9e1cd2cae488 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78042025-01-15T02:37:53.902ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2291 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2d5eaafc-9584-4c16-8174-9e1cd2cae488 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78052025-01-15T02:37:54.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 906985db-a8cf-4896-9688-93d66b9ec2d5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78062025-01-15T02:37:54.923ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 906985db-a8cf-4896-9688-93d66b9ec2d5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78072025-01-15T02:37:54.925ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2698 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 906985db-a8cf-4896-9688-93d66b9ec2d5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78082025-01-15T02:37:55.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 422 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = e32d78c1-66d5-4f44-b8c5-b5e31d8e7b97 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
78092025-01-15T02:37:55.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f0092ce-fdf9-48cc-81d7-3acacc61f818 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78102025-01-15T02:37:55.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f0092ce-fdf9-48cc-81d7-3acacc61f818 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78112025-01-15T02:37:55.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2722 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6f0092ce-fdf9-48cc-81d7-3acacc61f818 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78122025-01-15T02:37:56.444ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b96f8bb7-1681-41cc-abf7-e3884bb37e7a uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-01-15T02:37:56.427218126Z }
78132025-01-15T02:37:56.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b96f8bb7-1681-41cc-abf7-e3884bb37e7a roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78142025-01-15T02:37:56.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b96f8bb7-1681-41cc-abf7-e3884bb37e7a resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78152025-01-15T02:37:56.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72616 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b96f8bb7-1681-41cc-abf7-e3884bb37e7a response_code = 204 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78162025-01-15T02:37:56.500ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5c43024c-b037-4e2f-ac8e-20ca5d7e0ddb uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-01-15T02:37:56.500370010Z }
78172025-01-15T02:37:56.500ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5c43024c-b037-4e2f-ac8e-20ca5d7e0ddb roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78182025-01-15T02:37:56.500ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5c43024c-b037-4e2f-ac8e-20ca5d7e0ddb resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78192025-01-15T02:37:56.511ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11228 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5c43024c-b037-4e2f-ac8e-20ca5d7e0ddb response_code = 204 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78202025-01-15T02:37:56.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d53a249a-3df1-4ceb-b104-69a2889a3ec6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78212025-01-15T02:37:56.967ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d53a249a-3df1-4ceb-b104-69a2889a3ec6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78222025-01-15T02:37:56.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1745 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d53a249a-3df1-4ceb-b104-69a2889a3ec6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78232025-01-15T02:37:57.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 153cf708-b37d-4b9f-8192-7086a3728157 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78242025-01-15T02:37:57.990ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 153cf708-b37d-4b9f-8192-7086a3728157 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78252025-01-15T02:37:57.992ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2571 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 153cf708-b37d-4b9f-8192-7086a3728157 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78262025-01-15T02:37:59.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 97fe971c-89a2-47b3-af33-4733d38c72c8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78272025-01-15T02:37:59.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 97fe971c-89a2-47b3-af33-4733d38c72c8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78282025-01-15T02:37:59.017ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2490 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 97fe971c-89a2-47b3-af33-4733d38c72c8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78292025-01-15T02:38:00.038ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b7d86d01-4882-477f-8657-b65778ce6a4f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78302025-01-15T02:38:00.038ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b7d86d01-4882-477f-8657-b65778ce6a4f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78312025-01-15T02:38:00.040ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b7d86d01-4882-477f-8657-b65778ce6a4f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78322025-01-15T02:38:00.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
78332025-01-15T02:38:00.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
78342025-01-15T02:38:00.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78352025-01-15T02:38:00.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78362025-01-15T02:38:00.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78372025-01-15T02:38:00.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78382025-01-15T02:38:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78392025-01-15T02:38:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78402025-01-15T02:38:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78412025-01-15T02:38:00.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.994508ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
78422025-01-15T02:38:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
78432025-01-15T02:38:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78442025-01-15T02:38:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78452025-01-15T02:38:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78462025-01-15T02:38:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78472025-01-15T02:38:00.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78482025-01-15T02:38:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
78492025-01-15T02:38:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78502025-01-15T02:38:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78512025-01-15T02:38:00.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
78522025-01-15T02:38:00.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.882597ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
78532025-01-15T02:38:01.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d2eb44a2-eea5-462c-8705-9311f7d83ced roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78542025-01-15T02:38:01.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d2eb44a2-eea5-462c-8705-9311f7d83ced resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78552025-01-15T02:38:01.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2319 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d2eb44a2-eea5-462c-8705-9311f7d83ced response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78562025-01-15T02:38:02.088ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e540a0dc-8bf1-4809-a2c7-0d93ff88e059 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78572025-01-15T02:38:02.088ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e540a0dc-8bf1-4809-a2c7-0d93ff88e059 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78582025-01-15T02:38:02.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2620 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e540a0dc-8bf1-4809-a2c7-0d93ff88e059 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78592025-01-15T02:38:03.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5fd41998-1728-40a3-a6db-88b779ee43d5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78602025-01-15T02:38:03.110ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5fd41998-1728-40a3-a6db-88b779ee43d5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78612025-01-15T02:38:03.112ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2293 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5fd41998-1728-40a3-a6db-88b779ee43d5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78622025-01-15T02:38:04.131ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5f3a3e64-b9c9-4a22-a79a-bc6af9425e2e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78632025-01-15T02:38:04.131ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5f3a3e64-b9c9-4a22-a79a-bc6af9425e2e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78642025-01-15T02:38:04.133ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5f3a3e64-b9c9-4a22-a79a-bc6af9425e2e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78652025-01-15T02:38:05.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 48663a0d-7cd7-438b-9c2b-79c0aa1f1c5c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78662025-01-15T02:38:05.153ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 48663a0d-7cd7-438b-9c2b-79c0aa1f1c5c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78672025-01-15T02:38:05.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2246 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 48663a0d-7cd7-438b-9c2b-79c0aa1f1c5c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78682025-01-15T02:38:05.746ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 562 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 0753e241-168d-47c5-9c4d-72eaf1f427e8 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
78692025-01-15T02:38:06.175ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 977a5d43-420b-411a-b106-ce54de24d69e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78702025-01-15T02:38:06.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 977a5d43-420b-411a-b106-ce54de24d69e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78712025-01-15T02:38:06.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 977a5d43-420b-411a-b106-ce54de24d69e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78722025-01-15T02:38:07.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 70d437f4-bc2d-4f8b-9538-d8d99f23c1f6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78732025-01-15T02:38:07.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 70d437f4-bc2d-4f8b-9538-d8d99f23c1f6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78742025-01-15T02:38:07.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2604 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 70d437f4-bc2d-4f8b-9538-d8d99f23c1f6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78752025-01-15T02:38:08.222ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 12ad7315-7025-4978-bea9-b8f85d3cee2e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78762025-01-15T02:38:08.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 12ad7315-7025-4978-bea9-b8f85d3cee2e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78772025-01-15T02:38:08.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 12ad7315-7025-4978-bea9-b8f85d3cee2e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78782025-01-15T02:38:09.245ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7c129e5-ada7-4490-b790-1e8cc64eb819 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78792025-01-15T02:38:09.245ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7c129e5-ada7-4490-b790-1e8cc64eb819 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78802025-01-15T02:38:09.247ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2827 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7c129e5-ada7-4490-b790-1e8cc64eb819 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78812025-01-15T02:38:09.690ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fd464929-136c-4eb6-a868-4d93427c623b uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-01-15T02:38:09.689911399Z }
78822025-01-15T02:38:09.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fd464929-136c-4eb6-a868-4d93427c623b roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78832025-01-15T02:38:09.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fd464929-136c-4eb6-a868-4d93427c623b resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78842025-01-15T02:38:09.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 18727 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fd464929-136c-4eb6-a868-4d93427c623b response_code = 204 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
78852025-01-15T02:38:10.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6aff9920-de83-449a-b239-a35b32fcac9d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78862025-01-15T02:38:10.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6aff9920-de83-449a-b239-a35b32fcac9d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78872025-01-15T02:38:10.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1892 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6aff9920-de83-449a-b239-a35b32fcac9d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
78882025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
78892025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
78902025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
78912025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
78922025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78932025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
78942025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
78952025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
78962025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
78972025-01-15T02:38:10.503ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:28:10.503199055 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
78982025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
78992025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79002025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
79012025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
79022025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
79032025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79042025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
79052025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
79062025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
79072025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
79082025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
79092025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
79102025-01-15T02:38:10.503ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
79112025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
79122025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.93\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
79132025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79142025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
79152025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
79162025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
79172025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
79182025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
79192025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
79202025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79212025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
79222025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
79232025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
79242025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79252025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79262025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
79272025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
79282025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
79292025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
79302025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
79312025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
79322025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
79332025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79342025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
79352025-01-15T02:38:10.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
79362025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
79372025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
79382025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
79392025-01-15T02:38:10.503ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
79402025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79412025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
79422025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
79432025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79442025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
79452025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
79462025-01-15T02:38:10.503ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 3977, tv_nsec: 341971872 } })
79472025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
79482025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
79492025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
79502025-01-15T02:38:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
79512025-01-15T02:38:10.503ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 3977, tv_nsec: 342026934 } })
79522025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79532025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79542025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79552025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
79562025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79572025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
79582025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
79592025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79602025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
79612025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
79622025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.220483ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
79632025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
79642025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
79652025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
79662025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
79672025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
79682025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
79692025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
79702025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
79712025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
79722025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
79732025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
79742025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
79752025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
79762025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.48516ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
79772025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
79782025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
79792025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
79802025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
79812025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
79822025-01-15T02:38:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
79832025-01-15T02:38:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79842025-01-15T02:38:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79852025-01-15T02:38:10.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79862025-01-15T02:38:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79872025-01-15T02:38:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79882025-01-15T02:38:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.31753ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
79892025-01-15T02:38:10.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3977, tv_nsec: 345000464 } })
79902025-01-15T02:38:10.507ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
79912025-01-15T02:38:10.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
79922025-01-15T02:38:10.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79932025-01-15T02:38:10.508ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:28:10.503199055 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
79942025-01-15T02:38:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
79952025-01-15T02:38:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.856041ms iteration = 12 status = {"expiration":"2025-01-15T02:28:10.503199055Z","pruned":[]}
79962025-01-15T02:38:10.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79972025-01-15T02:38:10.510ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3977, tv_nsec: 348258972 } })
79982025-01-15T02:38:10.510ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
79992025-01-15T02:38:10.510ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80002025-01-15T02:38:10.511ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80012025-01-15T02:38:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
80022025-01-15T02:38:10.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80032025-01-15T02:38:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80042025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
80052025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80062025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
80072025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
80082025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80092025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
80102025-01-15T02:38:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80112025-01-15T02:38:10.515ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80122025-01-15T02:38:10.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80132025-01-15T02:38:10.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80142025-01-15T02:38:10.516ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80152025-01-15T02:38:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80162025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
80172025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80182025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80192025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
80202025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80212025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
80222025-01-15T02:38:10.518ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80232025-01-15T02:38:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
80242025-01-15T02:38:10.519ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80252025-01-15T02:38:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80262025-01-15T02:38:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
80272025-01-15T02:38:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80282025-01-15T02:38:10.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80292025-01-15T02:38:10.522ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80302025-01-15T02:38:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80312025-01-15T02:38:10.522ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.081138ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
80322025-01-15T02:38:10.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
80332025-01-15T02:38:10.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80342025-01-15T02:38:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80352025-01-15T02:38:10.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80362025-01-15T02:38:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80372025-01-15T02:38:10.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80382025-01-15T02:38:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80392025-01-15T02:38:10.529ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3977, tv_nsec: 367304346 } })
80402025-01-15T02:38:10.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80412025-01-15T02:38:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80422025-01-15T02:38:10.529ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
80432025-01-15T02:38:10.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80442025-01-15T02:38:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80452025-01-15T02:38:10.531ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80462025-01-15T02:38:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80472025-01-15T02:38:10.533ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80482025-01-15T02:38:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
80492025-01-15T02:38:10.537ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80502025-01-15T02:38:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80512025-01-15T02:38:10.557ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80522025-01-15T02:38:10.557ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
80532025-01-15T02:38:10.557ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80542025-01-15T02:38:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80552025-01-15T02:38:10.559ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80562025-01-15T02:38:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
80572025-01-15T02:38:10.562ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80582025-01-15T02:38:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80592025-01-15T02:38:10.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80602025-01-15T02:38:10.566ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80612025-01-15T02:38:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
80622025-01-15T02:38:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
80632025-01-15T02:38:10.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80642025-01-15T02:38:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80652025-01-15T02:38:10.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.22578ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
80662025-01-15T02:38:10.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 64.544002ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
80672025-01-15T02:38:10.575ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 70.588264ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
80682025-01-15T02:38:10.575ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
80692025-01-15T02:38:10.575ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80702025-01-15T02:38:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
80712025-01-15T02:38:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80722025-01-15T02:38:10.575ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80732025-01-15T02:38:10.576ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
80742025-01-15T02:38:10.576ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
80752025-01-15T02:38:10.576ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 71.545571ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
80762025-01-15T02:38:10.576ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
80772025-01-15T02:38:10.576ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
80782025-01-15T02:38:10.576ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 73.346899ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
80792025-01-15T02:38:10.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80802025-01-15T02:38:10.578ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80812025-01-15T02:38:10.578ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80822025-01-15T02:38:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80832025-01-15T02:38:10.582ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80842025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
80852025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80862025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
80872025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80882025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
80892025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80902025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80912025-01-15T02:38:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
80922025-01-15T02:38:10.583ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 79.995899ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
80932025-01-15T02:38:10.583ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
80942025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
80952025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
80962025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80972025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80982025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
80992025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
81002025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
81012025-01-15T02:38:10.584ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81022025-01-15T02:38:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81032025-01-15T02:38:10.585ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81042025-01-15T02:38:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81052025-01-15T02:38:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 83.070222ms iteration = 26 status = {"physical_disks_added":0}
81062025-01-15T02:38:10.596ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
81072025-01-15T02:38:10.596ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.01212ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
81082025-01-15T02:38:10.597ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
81092025-01-15T02:38:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
81102025-01-15T02:38:10.618ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
81112025-01-15T02:38:10.618ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
81122025-01-15T02:38:10.618ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 115.076046ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
81132025-01-15T02:38:10.619ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.210799ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
81142025-01-15T02:38:10.624ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81152025-01-15T02:38:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
81162025-01-15T02:38:10.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81172025-01-15T02:38:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81182025-01-15T02:38:10.628ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
81192025-01-15T02:38:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81202025-01-15T02:38:10.630ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 125.904957ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
81212025-01-15T02:38:10.646ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81222025-01-15T02:38:10.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81232025-01-15T02:38:10.663ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
81242025-01-15T02:38:10.684ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
81252025-01-15T02:38:10.684ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
81262025-01-15T02:38:10.685ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3e73162-919a-48dc-8e6c-b0b674f12fbb", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
81272025-01-15T02:38:10.685ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
81282025-01-15T02:38:10.686ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
81292025-01-15T02:38:10.711ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
81302025-01-15T02:38:10.711ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
81312025-01-15T02:38:10.712ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "046c3484-fcb3-45fb-b573-c6d47a8b74ea", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
81322025-01-15T02:38:10.712ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
81332025-01-15T02:38:10.714ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
81342025-01-15T02:38:10.784ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
81352025-01-15T02:38:10.787ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/04760650-1e85-40f1-9f70-557a11b03e88/state
81362025-01-15T02:38:10.792ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81372025-01-15T02:38:10.794ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
81382025-01-15T02:38:10.794ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
81392025-01-15T02:38:10.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81402025-01-15T02:38:10.810ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
81412025-01-15T02:38:10.810ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
81422025-01-15T02:38:10.810ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "71491498-6be3-4bf4-9f41-9da52f75802e", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
81432025-01-15T02:38:10.810ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
81442025-01-15T02:38:10.812ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
81452025-01-15T02:38:10.821ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81462025-01-15T02:38:10.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
81472025-01-15T02:38:10.824ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81482025-01-15T02:38:10.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81492025-01-15T02:38:10.830ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81502025-01-15T02:38:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
81512025-01-15T02:38:10.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81522025-01-15T02:38:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81532025-01-15T02:38:10.835ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
81542025-01-15T02:38:10.835ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 331.706731ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
81552025-01-15T02:38:10.835ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
81562025-01-15T02:38:10.835ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 332.366599ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
81572025-01-15T02:38:10.843ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81582025-01-15T02:38:10.849ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
81592025-01-15T02:38:10.849ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81602025-01-15T02:38:10.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81612025-01-15T02:38:10.852ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 349.115781ms iteration = 23 status = 0
81622025-01-15T02:38:10.862ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
81632025-01-15T02:38:10.862ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
81642025-01-15T02:38:10.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81652025-01-15T02:38:10.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81662025-01-15T02:38:10.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81672025-01-15T02:38:10.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
81682025-01-15T02:38:10.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81692025-01-15T02:38:10.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81702025-01-15T02:38:10.869ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
81712025-01-15T02:38:10.869ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
81722025-01-15T02:38:10.869ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81732025-01-15T02:38:10.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81742025-01-15T02:38:10.870ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
81752025-01-15T02:38:10.870ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
81762025-01-15T02:38:10.870ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
81772025-01-15T02:38:10.870ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 366.968862ms iteration = 12 status = {"generation":1}
81782025-01-15T02:38:10.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81792025-01-15T02:38:10.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
81802025-01-15T02:38:10.876ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81812025-01-15T02:38:10.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81822025-01-15T02:38:10.882ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
81832025-01-15T02:38:10.882ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
81842025-01-15T02:38:10.882ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
81852025-01-15T02:38:10.882ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 379.628134ms iteration = 12 status = {"generation":2}
81862025-01-15T02:38:10.904ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
81872025-01-15T02:38:10.904ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 401.282589ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
81882025-01-15T02:38:10.910ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81892025-01-15T02:38:10.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81902025-01-15T02:38:10.912ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81912025-01-15T02:38:10.914ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81922025-01-15T02:38:10.914ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81932025-01-15T02:38:10.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81942025-01-15T02:38:10.969ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
81952025-01-15T02:38:10.981ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
81962025-01-15T02:38:10.981ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ab7fb4e1-4b31-4132-ae26-28d2bcade630", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
81972025-01-15T02:38:10.981ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
81982025-01-15T02:38:10.983ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e2eba231-274b-48ee-9375-f5ffe7688aff", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
81992025-01-15T02:38:10.983ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
82002025-01-15T02:38:10.984ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
82012025-01-15T02:38:10.984ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 481.501063ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
82022025-01-15T02:38:10.998ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 495.092251ms iteration = 23 status = {}
82032025-01-15T02:38:11.066ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
82042025-01-15T02:38:11.066ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a7aee1f-0547-4d42-90e1-345544017e10", "content-length": "2", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82052025-01-15T02:38:11.069ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
82062025-01-15T02:38:11.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
82072025-01-15T02:38:11.084ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
82082025-01-15T02:38:11.084ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a4c9e848-7e54-42ff-83a3-6eda537d41c2", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82092025-01-15T02:38:11.084ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82102025-01-15T02:38:11.085ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
82112025-01-15T02:38:11.085ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
82122025-01-15T02:38:11.085ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "6ea9d30f-8b1e-4788-8b5a-239636855a18", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82132025-01-15T02:38:11.085ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82142025-01-15T02:38:11.087ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82152025-01-15T02:38:11.087ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3cf092d1-480c-4ee3-a360-2f9ad87e5cb4", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82162025-01-15T02:38:11.087ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82172025-01-15T02:38:11.088ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 585.069017ms iteration = 23 status = null
82182025-01-15T02:38:11.088ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
82192025-01-15T02:38:11.088ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82202025-01-15T02:38:11.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82212025-01-15T02:38:11.089ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
82222025-01-15T02:38:11.089ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 586.308751ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
82232025-01-15T02:38:11.117ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82242025-01-15T02:38:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82252025-01-15T02:38:11.119ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
82262025-01-15T02:38:11.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82272025-01-15T02:38:11.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82282025-01-15T02:38:11.129ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
82292025-01-15T02:38:11.129ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "f32f3eac-45c6-4556-9c43-32b78beb6e9f", "content-length": "134", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82302025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
82312025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
82322025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
82332025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
82342025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82352025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82362025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82372025-01-15T02:38:11.129ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
82382025-01-15T02:38:11.130ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82392025-01-15T02:38:11.130ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
82402025-01-15T02:38:11.130ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
82412025-01-15T02:38:11.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82422025-01-15T02:38:11.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82432025-01-15T02:38:11.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
82442025-01-15T02:38:11.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "73a2840f-7349-4f35-94fe-d6bc076e31c2", "content-length": "2", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82452025-01-15T02:38:11.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
82462025-01-15T02:38:11.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "2915b8c4-3770-4f2b-bc19-4f43c3761c7f", "content-length": "52", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82472025-01-15T02:38:11.133ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
82482025-01-15T02:38:11.133ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
82492025-01-15T02:38:11.133ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
82502025-01-15T02:38:11.148ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cb64395-5a7f-4e3e-9f29-387707873e39", "content-length": "583", "date": "Wed, 15 Jan 2025 02:38:11 GMT"} })
82512025-01-15T02:38:11.148ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82522025-01-15T02:38:11.148ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8d938d8-9520-4fc7-8d06-e566ea57b942", "content-length": "810", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82532025-01-15T02:38:11.148ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(3887612) }} sled = sock
82542025-01-15T02:38:11.148ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
82552025-01-15T02:38:11.148ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
82562025-01-15T02:38:11.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82572025-01-15T02:38:11.149ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 646.428964ms iteration = 23 status = {}
82582025-01-15T02:38:11.175ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
82592025-01-15T02:38:11.176ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
82602025-01-15T02:38:11.191ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {8bfad461-5ff5-4bae-8dc5-5c05bfcee1cf: {10.151.2.176: {1187a63a-5e12-4ea5-9717-553d241f7bb3}, 10.151.2.175: {1187a63a-5e12-4ea5-9717-553d241f7bb3}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
82612025-01-15T02:38:11.191ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
82622025-01-15T02:38:11.192ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "92b63ef1-1907-4f6f-b1a8-2634d7ffc78c", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82632025-01-15T02:38:11.192ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82642025-01-15T02:38:11.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82652025-01-15T02:38:11.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82662025-01-15T02:38:11.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82672025-01-15T02:38:11.197ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82682025-01-15T02:38:11.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82692025-01-15T02:38:11.201ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82702025-01-15T02:38:11.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82712025-01-15T02:38:11.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82722025-01-15T02:38:11.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82732025-01-15T02:38:11.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82742025-01-15T02:38:11.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82752025-01-15T02:38:11.207ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
82762025-01-15T02:38:11.207ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:34794
82772025-01-15T02:38:11.207ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34794 req_id = 1bb0d0c4-fd48-4bcb-8ed2-eb9623dc35ea roles = RoleSet { roles: {} } uri = /metrics/producers
82782025-01-15T02:38:11.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34794 req_id = 1bb0d0c4-fd48-4bcb-8ed2-eb9623dc35ea resource = Database result = Ok(()) uri = /metrics/producers
82792025-01-15T02:38:11.209ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
82802025-01-15T02:38:11.209ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
82812025-01-15T02:38:11.209ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
82822025-01-15T02:38:11.213ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "aba31e28-6276-4c3b-82ca-5e6538a88754", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82832025-01-15T02:38:11.215ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 712.202578ms iteration = 23 status = {}
82842025-01-15T02:38:11.246ZINFOnexus: assigned collector to new producer collector_id = f71d33aa-7320-48b9-81d2-899a100e78cd file = nexus/src/app/oximeter.rs:76 producer_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
82852025-01-15T02:38:11.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38615 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34794 req_id = 1bb0d0c4-fd48-4bcb-8ed2-eb9623dc35ea response_code = 201 uri = /metrics/producers
82862025-01-15T02:38:11.246ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "1bb0d0c4-fd48-4bcb-8ed2-eb9623dc35ea", "content-length": "41", "date": "Wed, 15 Jan 2025 02:38:11 GMT"} })
82872025-01-15T02:38:11.248ZDEBGnexus (producer-registration-task): registered with nexus successfully
82882025-01-15T02:38:11.248ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
82892025-01-15T02:38:11.261ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e9b3f284-bb50-46d6-8b86-02bcc02850a6", "date": "Wed, 15 Jan 2025 02:38:11 GMT"} })
82902025-01-15T02:38:11.261ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
82912025-01-15T02:38:11.261ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
82922025-01-15T02:38:11.261ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf83a2d6-9431-40e5-9593-20567d29d1f3", "content-length": "682", "date": "Wed, 15 Jan 2025 02:38:10 GMT"} })
82932025-01-15T02:38:11.261ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82942025-01-15T02:38:11.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82952025-01-15T02:38:11.265ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82962025-01-15T02:38:11.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82972025-01-15T02:38:11.269ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82982025-01-15T02:38:11.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82992025-01-15T02:38:11.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83002025-01-15T02:38:11.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83012025-01-15T02:38:11.279ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 775.668781ms iteration = 23 status = {}
83022025-01-15T02:38:11.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7af4f85-39d2-47ab-a3ce-32cb43d61a4c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
83032025-01-15T02:38:11.326ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7af4f85-39d2-47ab-a3ce-32cb43d61a4c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
83042025-01-15T02:38:11.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2250 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c7af4f85-39d2-47ab-a3ce-32cb43d61a4c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
83052025-01-15T02:38:12.180ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-15T02:38:12.180352232Z }
83062025-01-15T02:38:12.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83072025-01-15T02:38:12.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83082025-01-15T02:38:12.225ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_needs_update = true
83092025-01-15T02:38:12.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83102025-01-15T02:38:12.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83112025-01-15T02:38:12.228ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83122025-01-15T02:38:12.228ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83132025-01-15T02:38:12.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83142025-01-15T02:38:12.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83152025-01-15T02:38:12.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83162025-01-15T02:38:12.233ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83172025-01-15T02:38:12.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83182025-01-15T02:38:12.267ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83192025-01-15T02:38:12.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83202025-01-15T02:38:12.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83212025-01-15T02:38:12.271ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83222025-01-15T02:38:12.274ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, lookup_type: ById(25a10c65-c21a-4403-8acf-3f8c60b10e2b) }, key: 13f9223c-15ec-40ed-8ceb-79dbb1e92fe0, lookup_type: ById(13f9223c-15ec-40ed-8ceb-79dbb1e92fe0) }, key: 62d616e1-f148-4a1c-90a4-4e962236db27, lookup_type: ById(62d616e1-f148-4a1c-90a4-4e962236db27) } result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83232025-01-15T02:38:12.286ZINFOnexus (dropshot_internal): starting update saga for 62d616e1-f148-4a1c-90a4-4e962236db27 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1544 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-15T02:38:12.180352232Z }
83242025-01-15T02:38:12.286ZDEBGnexus (dropshot_internal): preparing instance-update saga for 62d616e1-f148-4a1c-90a4-4e962236db27... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83252025-01-15T02:38:12.286ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106322 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bd53a6ca-1a29-4edd-aec5-c78bed4fd9f3 response_code = 204 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83262025-01-15T02:38:12.287ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c02be74-541a-49d8-aea5-0c502b6ce0a9 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-15T02:38:12.180352232Z }
83272025-01-15T02:38:12.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c02be74-541a-49d8-aea5-0c502b6ce0a9 roles = RoleSet { roles: {} } uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83282025-01-15T02:38:12.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c02be74-541a-49d8-aea5-0c502b6ce0a9 resource = Database result = Ok(()) uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83292025-01-15T02:38:12.289ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update
83302025-01-15T02:38:12.291ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4515 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7c02be74-541a-49d8-aea5-0c502b6ce0a9 response_code = 204 uri = /vmms/04760650-1e85-40f1-9f70-557a11b03e88
83312025-01-15T02:38:12.292ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/04760650-1e85-40f1-9f70-557a11b03e88/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "edee903c-a0ae-4ee6-bb0d-659d2b72feaf", "content-length": "132", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} })
83322025-01-15T02:38:12.295ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 sled_id = ce66c06a-7d54-47a6-860d-59da68080fee state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-15T02:38:12.180352232Z }, migration_in: None, migration_out: None } vmm_id = 04760650-1e85-40f1-9f70-557a11b03e88
83332025-01-15T02:38:12.295ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2077 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 sled_id = ce66c06a-7d54-47a6-860d-59da68080fee vmm_id = 04760650-1e85-40f1-9f70-557a11b03e88 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-15T02:38:12.180352232Z }
83342025-01-15T02:38:12.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 roles = RoleSet { roles: {} } sled_id = ce66c06a-7d54-47a6-860d-59da68080fee vmm_id = 04760650-1e85-40f1-9f70-557a11b03e88
83352025-01-15T02:38:12.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 resource = Database result = Ok(()) sled_id = ce66c06a-7d54-47a6-860d-59da68080fee vmm_id = 04760650-1e85-40f1-9f70-557a11b03e88
83362025-01-15T02:38:12.299ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"62d616e1-f148-4a1c-90a4-4e962236db27","lookup_type":{"ById":"62d616e1-f148-4a1c-90a4-4e962236db27"},"parent":{"key":"13f9223c-15ec-40ed-8ceb-79dbb1e92fe0","lookup_type":{"ById":"13f9223c-15ec-40ed-8ceb-79dbb1e92fe0"},"parent":{"key":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","lookup_type":{"ById":"25a10c65-c21a-4403-8acf-3f8c60b10e2b"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83372025-01-15T02:38:12.299ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update
83382025-01-15T02:38:12.300ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
83392025-01-15T02:38:12.301ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.798387398s iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"destroyed":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
83402025-01-15T02:38:12.336ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update
83412025-01-15T02:38:12.336ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83422025-01-15T02:38:12.336ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83432025-01-15T02:38:12.336ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83442025-01-15T02:38:12.344ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83452025-01-15T02:38:12.344ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83462025-01-15T02:38:12.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46bbefea-9045-491a-9261-9f60f2be1833 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
83472025-01-15T02:38:12.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46bbefea-9045-491a-9261-9f60f2be1833 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
83482025-01-15T02:38:12.350ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83492025-01-15T02:38:12.350ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83502025-01-15T02:38:12.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46bbefea-9045-491a-9261-9f60f2be1833 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
83512025-01-15T02:38:12.364ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83522025-01-15T02:38:12.364ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("e5e073cc-7ea8-419d-9fab-eec8e98f6f4a")) node_id = 0 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83532025-01-15T02:38:12.371ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83542025-01-15T02:38:12.371ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83552025-01-15T02:38:12.392ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update
83562025-01-15T02:38:12.396ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83572025-01-15T02:38:12.397ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83582025-01-15T02:38:12.398ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83592025-01-15T02:38:12.399ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83602025-01-15T02:38:12.400ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83612025-01-15T02:38:12.400ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83622025-01-15T02:38:12.405ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83632025-01-15T02:38:12.405ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83642025-01-15T02:38:12.406ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83652025-01-15T02:38:12.407ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83662025-01-15T02:38:12.408ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83672025-01-15T02:38:12.408ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83682025-01-15T02:38:12.410ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83692025-01-15T02:38:12.413ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, lookup_type: ById(25a10c65-c21a-4403-8acf-3f8c60b10e2b) }, key: 13f9223c-15ec-40ed-8ceb-79dbb1e92fe0, lookup_type: ById(13f9223c-15ec-40ed-8ceb-79dbb1e92fe0) }, key: 62d616e1-f148-4a1c-90a4-4e962236db27, lookup_type: ById(62d616e1-f148-4a1c-90a4-4e962236db27) } result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83702025-01-15T02:38:12.413ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance updater_id = e5e073cc-7ea8-419d-9fab-eec8e98f6f4a
83712025-01-15T02:38:12.413ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83722025-01-15T02:38:12.413ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83732025-01-15T02:38:12.426ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance updater_id = e5e073cc-7ea8-419d-9fab-eec8e98f6f4a
83742025-01-15T02:38:12.426ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83752025-01-15T02:38:12.427ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance
83762025-01-15T02:38:12.431ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 locked_gen = Generation(Generation(2)) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = LockInstance updater_id = e5e073cc-7ea8-419d-9fab-eec8e98f6f4a
83772025-01-15T02:38:12.431ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83782025-01-15T02:38:12.431ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("e5e073cc-7ea8-419d-9fab-eec8e98f6f4a")}) node_id = 1 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83792025-01-15T02:38:12.437ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83802025-01-15T02:38:12.437ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
83812025-01-15T02:38:12.448ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83822025-01-15T02:38:12.448ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83832025-01-15T02:38:12.449ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83842025-01-15T02:38:12.449ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83852025-01-15T02:38:12.450ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83862025-01-15T02:38:12.450ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83872025-01-15T02:38:12.452ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83882025-01-15T02:38:12.455ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, lookup_type: ById(25a10c65-c21a-4403-8acf-3f8c60b10e2b) }, key: 13f9223c-15ec-40ed-8ceb-79dbb1e92fe0, lookup_type: ById(13f9223c-15ec-40ed-8ceb-79dbb1e92fe0) }, key: 62d616e1-f148-4a1c-90a4-4e962236db27, lookup_type: ById(62d616e1-f148-4a1c-90a4-4e962236db27) } result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83892025-01-15T02:38:12.455ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83902025-01-15T02:38:12.455ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
83912025-01-15T02:38:12.466ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 04760650-1e85-40f1-9f70-557a11b03e88, time_created: 2025-01-15T02:37:45.803528Z, time_deleted: None, instance_id: 62d616e1-f148-4a1c-90a4-4e962236db27, sled_id: ce66c06a-7d54-47a6-860d-59da68080fee, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-01-15T02:38:12.180352Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-01-15T02:37:40.633349Z, gen: Generation(Generation(3)), propolis_id: Some(04760650-1e85-40f1-9f70-557a11b03e88), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(04760650-1e85-40f1-9f70-557a11b03e88 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-01-15T02:38:12.466233346Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
83922025-01-15T02:38:12.466ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
83932025-01-15T02:38:12.466ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"62d616e1-f148-4a1c-90a4-4e962236db27","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"04760650-1e85-40f1-9f70-557a11b03e88"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"62d616e1-f148-4a1c-90a4-4e962236db27","lookup_type":{"ById":"62d616e1-f148-4a1c-90a4-4e962236db27"},"parent":{"key":"13f9223c-15ec-40ed-8ceb-79dbb1e92fe0","lookup_type":{"ById":"13f9223c-15ec-40ed-8ceb-79dbb1e92fe0"},"parent":{"key":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","lookup_type":{"ById":"25a10c65-c21a-4403-8acf-3f8c60b10e2b"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"e5e073cc-7ea8-419d-9fab-eec8e98f6f4a"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"13f9223c-15ec-40ed-8ceb-79dbb1e92fe0","ram_diff":1073741824},"destroy_active_vmm":"04760650-1e85-40f1-9f70-557a11b03e88","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-01-15T02:38:12.466233346Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83942025-01-15T02:38:12.466ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
83952025-01-15T02:38:12.499ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
83962025-01-15T02:38:12.499ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83972025-01-15T02:38:12.499ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
83982025-01-15T02:38:12.499ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 047da118-4757-40e1-b880-83205f1094f2
83992025-01-15T02:38:12.504ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84002025-01-15T02:38:12.504ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84012025-01-15T02:38:12.509ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84022025-01-15T02:38:12.509ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84032025-01-15T02:38:12.513ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84042025-01-15T02:38:12.513ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("0c5b2e45-64af-4f7f-a873-0cd4ba493a68")) node_id = 0 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84052025-01-15T02:38:12.518ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84062025-01-15T02:38:12.518ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84072025-01-15T02:38:12.524ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 parent_lock = UpdaterLock { updater_id: e5e073cc-7ea8-419d-9fab-eec8e98f6f4a, locked_gen: Generation(Generation(2)) } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84082025-01-15T02:38:12.524ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = BecomeUpdater
84092025-01-15T02:38:12.524ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = BecomeUpdater
84102025-01-15T02:38:12.535ZDEBGnexus (SagaExecutor): inherited lock from e5e073cc-7ea8-419d-9fab-eec8e98f6f4a to 0c5b2e45-64af-4f7f-a873-0cd4ba493a68 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = e5e073cc-7ea8-419d-9fab-eec8e98f6f4a saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = BecomeUpdater updater_id = 0c5b2e45-64af-4f7f-a873-0cd4ba493a68
84112025-01-15T02:38:12.535ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84122025-01-15T02:38:12.535ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84132025-01-15T02:38:12.535ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("0c5b2e45-64af-4f7f-a873-0cd4ba493a68")}) node_id = 1 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84142025-01-15T02:38:12.541ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84152025-01-15T02:38:12.541ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84162025-01-15T02:38:12.544ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84172025-01-15T02:38:12.544ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84182025-01-15T02:38:12.547ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84192025-01-15T02:38:12.548ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84202025-01-15T02:38:12.553ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84212025-01-15T02:38:12.556ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27
84222025-01-15T02:38:12.556ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UpdateNetworkConfig
84232025-01-15T02:38:12.556ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UpdateNetworkConfig
84242025-01-15T02:38:12.559ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UpdateNetworkConfig
84252025-01-15T02:38:12.559ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UpdateNetworkConfig
84262025-01-15T02:38:12.563ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
84272025-01-15T02:38:12.563ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UpdateNetworkConfig
84282025-01-15T02:38:12.563ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UpdateNetworkConfig
84292025-01-15T02:38:12.565ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
84302025-01-15T02:38:12.565ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
84312025-01-15T02:38:12.566ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
84322025-01-15T02:38:12.567ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
84332025-01-15T02:38:12.568ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
84342025-01-15T02:38:12.568ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
84352025-01-15T02:38:12.568ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
84362025-01-15T02:38:12.577ZDEBGnexus: notifying dendrite of updates instance_id = Some(62d616e1-f148-4a1c-90a4-4e962236db27 (instance)) switch = switch0
84372025-01-15T02:38:12.577ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3979, tv_nsec: 415736941 } })
84382025-01-15T02:38:12.577ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
84392025-01-15T02:38:12.607ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
84402025-01-15T02:38:12.607ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
84412025-01-15T02:38:12.608ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "42956feb-4bbb-43da-a058-21b68522a83e", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} })
84422025-01-15T02:38:12.608ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
84432025-01-15T02:38:12.608ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
84442025-01-15T02:38:12.637ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
84452025-01-15T02:38:12.637ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "c63a5da3-ce1e-4050-9c65-43eca31d8417", "content-length": "4", "date": "Wed, 15 Jan 2025 02:38:11 GMT"} })
84462025-01-15T02:38:12.638ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84472025-01-15T02:38:12.638ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84482025-01-15T02:38:12.641ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84492025-01-15T02:38:12.641ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84502025-01-15T02:38:12.647ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
84512025-01-15T02:38:12.648ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
84522025-01-15T02:38:12.670ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-01-15T02:38:12.650829Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 13f9223c-15ec-40ed-8ceb-79dbb1e92fe0, time_modified: Some(2025-01-15T02:38:12.650829Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, time_modified: Some(2025-01-15T02:38:12.650829Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84532025-01-15T02:38:12.670ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84542025-01-15T02:38:12.670ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84552025-01-15T02:38:12.676ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84562025-01-15T02:38:12.676ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84572025-01-15T02:38:12.680ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84582025-01-15T02:38:12.680ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UnassignOximeterProducer
84592025-01-15T02:38:12.680ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UnassignOximeterProducer
84602025-01-15T02:38:12.690ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = f71d33aa-7320-48b9-81d2-899a100e78cd producer_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84612025-01-15T02:38:12.690ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UnassignOximeterProducer
84622025-01-15T02:38:12.690ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = UnassignOximeterProducer
84632025-01-15T02:38:12.730ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84642025-01-15T02:38:12.733ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/62d616e1-f148-4a1c-90a4-4e962236db27
84652025-01-15T02:38:12.752ZDEBGnexus (SagaExecutor): client response oximeter-collector = 62d616e1-f148-4a1c-90a4-4e962236db27 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/62d616e1-f148-4a1c-90a4-4e962236db27", status: 204, headers: {"x-request-id": "9ab5f9a7-a586-4bbe-97a6-51ac344b2616", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} }) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84662025-01-15T02:38:12.752ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = f71d33aa-7320-48b9-81d2-899a100e78cd producer_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84672025-01-15T02:38:12.753ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84682025-01-15T02:38:12.753ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84692025-01-15T02:38:12.760ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84702025-01-15T02:38:12.760ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84712025-01-15T02:38:12.766ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 lock = UpdaterLock { updater_id: 0c5b2e45-64af-4f7f-a873-0cd4ba493a68, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-01-15T02:38:12.466233346Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84722025-01-15T02:38:12.766ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = CommitInstanceUpdates
84732025-01-15T02:38:12.766ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = CommitInstanceUpdates
84742025-01-15T02:38:12.811ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 new_runtime = InstanceRuntimeState { time_updated: 2025-01-15T02:38:12.466233346Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
84752025-01-15T02:38:12.811ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84762025-01-15T02:38:12.811ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84772025-01-15T02:38:12.811ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
84782025-01-15T02:38:12.811ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
84792025-01-15T02:38:12.811ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
84802025-01-15T02:38:12.811ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84812025-01-15T02:38:12.811ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84822025-01-15T02:38:12.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84832025-01-15T02:38:12.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84842025-01-15T02:38:12.813ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84852025-01-15T02:38:12.815ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
84862025-01-15T02:38:12.815ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84872025-01-15T02:38:12.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84882025-01-15T02:38:12.816ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84892025-01-15T02:38:12.818ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84902025-01-15T02:38:12.818ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84912025-01-15T02:38:12.818ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
84922025-01-15T02:38:12.818ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84932025-01-15T02:38:12.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84942025-01-15T02:38:12.821ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84952025-01-15T02:38:12.821ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("62d616e1-f148-4a1c-90a4-4e962236db27"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("04760650-1e85-40f1-9f70-557a11b03e88")}) node_id = 7 saga_id = 047da118-4757-40e1-b880-83205f1094f2
84962025-01-15T02:38:12.824ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84972025-01-15T02:38:12.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84982025-01-15T02:38:12.827ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
84992025-01-15T02:38:12.827ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85002025-01-15T02:38:12.832ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85012025-01-15T02:38:12.832ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85022025-01-15T02:38:12.837ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85032025-01-15T02:38:12.837ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85042025-01-15T02:38:12.854ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
85052025-01-15T02:38:12.854ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
85062025-01-15T02:38:12.854ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "534317a8-3e0a-473f-a91c-f47d725fd5e4", "content-length": "372", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} })
85072025-01-15T02:38:12.854ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85082025-01-15T02:38:12.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85092025-01-15T02:38:12.858ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
85102025-01-15T02:38:12.858ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ReleaseSledResources
85112025-01-15T02:38:12.859ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ReleaseSledResources
85122025-01-15T02:38:12.860ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85132025-01-15T02:38:12.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
85142025-01-15T02:38:12.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85152025-01-15T02:38:12.863ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
85162025-01-15T02:38:12.863ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85172025-01-15T02:38:12.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
85182025-01-15T02:38:12.864ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85192025-01-15T02:38:12.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
85202025-01-15T02:38:12.866ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85212025-01-15T02:38:12.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
85222025-01-15T02:38:12.868ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {8bfad461-5ff5-4bae-8dc5-5c05bfcee1cf: {10.151.2.176: {1187a63a-5e12-4ea5-9717-553d241f7bb3}, 10.151.2.175: {1187a63a-5e12-4ea5-9717-553d241f7bb3}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
85232025-01-15T02:38:12.868ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
85242025-01-15T02:38:12.869ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "34fc51b6-cb26-497f-854a-3c8bdc4ce800", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} })
85252025-01-15T02:38:12.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85262025-01-15T02:38:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85272025-01-15T02:38:12.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85282025-01-15T02:38:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85292025-01-15T02:38:12.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85302025-01-15T02:38:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85312025-01-15T02:38:12.876ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
85322025-01-15T02:38:12.876ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
85332025-01-15T02:38:12.876ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ca00909d-ea39-4320-8383-98c18284985c", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} })
85342025-01-15T02:38:12.876ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85352025-01-15T02:38:12.876ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85362025-01-15T02:38:12.878ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 66.868354ms iteration = 24 status = {}
85372025-01-15T02:38:12.882ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85382025-01-15T02:38:12.882ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85392025-01-15T02:38:12.899ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
85402025-01-15T02:38:12.899ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "000e8c67-cf38-4a80-b8d1-48dec348b12a", "content-length": "691", "date": "Wed, 15 Jan 2025 02:38:12 GMT"} })
85412025-01-15T02:38:12.900ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
85422025-01-15T02:38:12.900ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
85432025-01-15T02:38:12.900ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
85442025-01-15T02:38:12.901ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 90.04113ms iteration = 24 status = {}
85452025-01-15T02:38:12.903ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 propolis_id = 04760650-1e85-40f1-9f70-557a11b03e88 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
85462025-01-15T02:38:12.903ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = MarkVmmDeleted
85472025-01-15T02:38:12.904ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = MarkVmmDeleted
85482025-01-15T02:38:12.919ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85492025-01-15T02:38:12.919ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85502025-01-15T02:38:12.923ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85512025-01-15T02:38:12.923ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85522025-01-15T02:38:12.927ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85532025-01-15T02:38:12.927ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85542025-01-15T02:38:12.931ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85552025-01-15T02:38:12.931ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85562025-01-15T02:38:12.937ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85572025-01-15T02:38:12.937ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85582025-01-15T02:38:12.938ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85592025-01-15T02:38:12.938ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85602025-01-15T02:38:12.940ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85612025-01-15T02:38:12.940ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85622025-01-15T02:38:12.941ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85632025-01-15T02:38:12.944ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, lookup_type: ById(25a10c65-c21a-4403-8acf-3f8c60b10e2b) }, key: 13f9223c-15ec-40ed-8ceb-79dbb1e92fe0, lookup_type: ById(13f9223c-15ec-40ed-8ceb-79dbb1e92fe0) }, key: 62d616e1-f148-4a1c-90a4-4e962236db27, lookup_type: ById(62d616e1-f148-4a1c-90a4-4e962236db27) } result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85642025-01-15T02:38:12.944ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85652025-01-15T02:38:12.944ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_node = ChainSuccessorSaga
85662025-01-15T02:38:12.950ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 karmic_state = WillReincarnate needs_reincarnation = false saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update
85672025-01-15T02:38:12.950ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85682025-01-15T02:38:12.950ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85692025-01-15T02:38:12.955ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85702025-01-15T02:38:12.955ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85712025-01-15T02:38:12.960ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85722025-01-15T02:38:12.960ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 047da118-4757-40e1-b880-83205f1094f2
85732025-01-15T02:38:12.966ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 047da118-4757-40e1-b880-83205f1094f2 sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85742025-01-15T02:38:12.966ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 047da118-4757-40e1-b880-83205f1094f2
85752025-01-15T02:38:12.976ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85762025-01-15T02:38:12.976ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 047da118-4757-40e1-b880-83205f1094f2 saga_name = instance-update saga_result = SagaResult { saga_id: 047da118-4757-40e1-b880-83205f1094f2, saga_log: SagaLog { saga_id: 047da118-4757-40e1-b880-83205f1094f2, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("0c5b2e45-64af-4f7f-a873-0cd4ba493a68")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("0c5b2e45-64af-4f7f-a873-0cd4ba493a68")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("62d616e1-f148-4a1c-90a4-4e962236db27"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("04760650-1e85-40f1-9f70-557a11b03e88")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("62d616e1-f148-4a1c-90a4-4e962236db27"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("04760650-1e85-40f1-9f70-557a11b03e88")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("0c5b2e45-64af-4f7f-a873-0cd4ba493a68"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("0c5b2e45-64af-4f7f-a873-0cd4ba493a68")}} }) }
85772025-01-15T02:38:12.976ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 047da118-4757-40e1-b880-83205f1094f2 instance_id = 62d616e1-f148-4a1c-90a4-4e962236db27 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update
85782025-01-15T02:38:12.976ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85792025-01-15T02:38:12.976ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
85802025-01-15T02:38:12.982ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85812025-01-15T02:38:12.982ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
85822025-01-15T02:38:12.988ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85832025-01-15T02:38:12.988ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
85842025-01-15T02:38:12.991ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85852025-01-15T02:38:12.991ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87
85862025-01-15T02:38:13.038ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update sec_id = b4bce764-527e-4a0d-9efa-46f5760fbb1d
85872025-01-15T02:38:13.038ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 8522ccfd-4441-460a-8aec-d14f76978d87 saga_name = start-instance-update saga_result = SagaResult { saga_id: 8522ccfd-4441-460a-8aec-d14f76978d87, saga_log: SagaLog { saga_id: 8522ccfd-4441-460a-8aec-d14f76978d87, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("e5e073cc-7ea8-419d-9fab-eec8e98f6f4a")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("e5e073cc-7ea8-419d-9fab-eec8e98f6f4a")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("e5e073cc-7ea8-419d-9fab-eec8e98f6f4a"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("e5e073cc-7ea8-419d-9fab-eec8e98f6f4a")}} }) }
85882025-01-15T02:38:13.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6566eb85-8774-4060-b7f2-e4a5382a6b8f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85892025-01-15T02:38:13.372ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6566eb85-8774-4060-b7f2-e4a5382a6b8f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85902025-01-15T02:38:13.374ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2402 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6566eb85-8774-4060-b7f2-e4a5382a6b8f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85912025-01-15T02:38:14.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ed7e317b-300a-4451-9e32-a6c8185cdf81 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85922025-01-15T02:38:14.395ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ed7e317b-300a-4451-9e32-a6c8185cdf81 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85932025-01-15T02:38:14.397ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2813 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ed7e317b-300a-4451-9e32-a6c8185cdf81 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85942025-01-15T02:38:15.421ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 322c9374-fe28-41e1-afef-ada904ac36a3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85952025-01-15T02:38:15.421ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 322c9374-fe28-41e1-afef-ada904ac36a3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85962025-01-15T02:38:15.424ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2767 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 322c9374-fe28-41e1-afef-ada904ac36a3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
85972025-01-15T02:38:15.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = fb0d67bb-ae08-40fe-933a-dbb3ebe4d547 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
85982025-01-15T02:38:15.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = fb0d67bb-ae08-40fe-933a-dbb3ebe4d547 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
85992025-01-15T02:38:15.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3210 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = fb0d67bb-ae08-40fe-933a-dbb3ebe4d547 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
86002025-01-15T02:38:15.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = d72457b6-be01-4ea4-9d1f-31c17c1265f8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
86012025-01-15T02:38:15.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = d72457b6-be01-4ea4-9d1f-31c17c1265f8 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
86022025-01-15T02:38:15.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2737 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = d72457b6-be01-4ea4-9d1f-31c17c1265f8 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
86032025-01-15T02:38:15.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 633 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 68dc7511-709c-4b86-aafb-f5e89793de32 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
86042025-01-15T02:38:16.443ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 956c0c64-f805-414d-aedb-99e72c442bc5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86052025-01-15T02:38:16.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 956c0c64-f805-414d-aedb-99e72c442bc5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86062025-01-15T02:38:16.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 956c0c64-f805-414d-aedb-99e72c442bc5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86072025-01-15T02:38:17.466ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46cfa2af-62b0-4e11-9d2f-503572e51019 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86082025-01-15T02:38:17.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46cfa2af-62b0-4e11-9d2f-503572e51019 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86092025-01-15T02:38:17.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46cfa2af-62b0-4e11-9d2f-503572e51019 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86102025-01-15T02:38:18.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8814846e-bd57-4703-b2f0-154045ca600d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86112025-01-15T02:38:18.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8814846e-bd57-4703-b2f0-154045ca600d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86122025-01-15T02:38:18.490ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2270 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8814846e-bd57-4703-b2f0-154045ca600d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86132025-01-15T02:38:19.510ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d68c07e-74f3-47c7-abb0-da8243690bb8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86142025-01-15T02:38:19.511ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d68c07e-74f3-47c7-abb0-da8243690bb8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86152025-01-15T02:38:19.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2249 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d68c07e-74f3-47c7-abb0-da8243690bb8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86162025-01-15T02:38:20.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
86172025-01-15T02:38:20.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
86182025-01-15T02:38:20.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86192025-01-15T02:38:20.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86202025-01-15T02:38:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86212025-01-15T02:38:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86222025-01-15T02:38:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86232025-01-15T02:38:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
86242025-01-15T02:38:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86252025-01-15T02:38:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86262025-01-15T02:38:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86272025-01-15T02:38:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86282025-01-15T02:38:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86292025-01-15T02:38:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
86302025-01-15T02:38:20.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86312025-01-15T02:38:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86322025-01-15T02:38:20.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
86332025-01-15T02:38:20.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.300642ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
86342025-01-15T02:38:20.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86352025-01-15T02:38:20.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86362025-01-15T02:38:20.532ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 960bce84-b877-4b36-bf29-3333077bc130 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86372025-01-15T02:38:20.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 960bce84-b877-4b36-bf29-3333077bc130 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86382025-01-15T02:38:20.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1678 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 960bce84-b877-4b36-bf29-3333077bc130 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86392025-01-15T02:38:20.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 54.011987ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86402025-01-15T02:38:21.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7fd0738-0d1c-42b3-9af7-6434ed1d4670 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86412025-01-15T02:38:21.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7fd0738-0d1c-42b3-9af7-6434ed1d4670 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86422025-01-15T02:38:21.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e7fd0738-0d1c-42b3-9af7-6434ed1d4670 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86432025-01-15T02:38:22.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6fdacadd-3def-4123-97ff-79f47339ce25 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86442025-01-15T02:38:22.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6fdacadd-3def-4123-97ff-79f47339ce25 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86452025-01-15T02:38:22.577ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2143 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6fdacadd-3def-4123-97ff-79f47339ce25 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86462025-01-15T02:38:23.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0cbb5152-e849-4a09-8a2a-1a24131446a7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86472025-01-15T02:38:23.598ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0cbb5152-e849-4a09-8a2a-1a24131446a7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86482025-01-15T02:38:23.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2237 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0cbb5152-e849-4a09-8a2a-1a24131446a7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86492025-01-15T02:38:24.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5bb9b042-d900-44c1-baff-81fffc30da27 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86502025-01-15T02:38:24.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5bb9b042-d900-44c1-baff-81fffc30da27 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86512025-01-15T02:38:24.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5bb9b042-d900-44c1-baff-81fffc30da27 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86522025-01-15T02:38:25.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88557d80-b94b-4718-a043-7a1c80cd2d1f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86532025-01-15T02:38:25.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88557d80-b94b-4718-a043-7a1c80cd2d1f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86542025-01-15T02:38:25.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 88557d80-b94b-4718-a043-7a1c80cd2d1f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86552025-01-15T02:38:25.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 546 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = ccc586f9-8561-4bcc-b7f3-64cec0c405dc response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
86562025-01-15T02:38:26.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 30a5d249-49dc-45aa-b4ee-e211b33915e6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86572025-01-15T02:38:26.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 30a5d249-49dc-45aa-b4ee-e211b33915e6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86582025-01-15T02:38:26.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 30a5d249-49dc-45aa-b4ee-e211b33915e6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86592025-01-15T02:38:27.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f61a81ed-9fb8-4ca0-b0af-7ecab7c9553b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86602025-01-15T02:38:27.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f61a81ed-9fb8-4ca0-b0af-7ecab7c9553b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86612025-01-15T02:38:27.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2610 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f61a81ed-9fb8-4ca0-b0af-7ecab7c9553b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86622025-01-15T02:38:28.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5518b0fd-557c-41ac-8624-12d017611514 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86632025-01-15T02:38:28.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5518b0fd-557c-41ac-8624-12d017611514 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86642025-01-15T02:38:28.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1931 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5518b0fd-557c-41ac-8624-12d017611514 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86652025-01-15T02:38:29.731ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b786e46-ad24-4a45-a4a2-15f99c75c52f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86662025-01-15T02:38:29.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b786e46-ad24-4a45-a4a2-15f99c75c52f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86672025-01-15T02:38:29.733ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1958 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1b786e46-ad24-4a45-a4a2-15f99c75c52f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86682025-01-15T02:38:30.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
86692025-01-15T02:38:30.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
86702025-01-15T02:38:30.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86712025-01-15T02:38:30.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86722025-01-15T02:38:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86732025-01-15T02:38:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86742025-01-15T02:38:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86752025-01-15T02:38:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
86762025-01-15T02:38:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86772025-01-15T02:38:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86782025-01-15T02:38:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86792025-01-15T02:38:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86802025-01-15T02:38:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86812025-01-15T02:38:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
86822025-01-15T02:38:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86832025-01-15T02:38:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86842025-01-15T02:38:30.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
86852025-01-15T02:38:30.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.435988ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
86862025-01-15T02:38:30.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86872025-01-15T02:38:30.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86882025-01-15T02:38:30.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.006842ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86892025-01-15T02:38:30.754ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7fc69603-5c5b-4132-a96e-abaa470a16f5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86902025-01-15T02:38:30.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7fc69603-5c5b-4132-a96e-abaa470a16f5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86912025-01-15T02:38:30.756ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1561 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7fc69603-5c5b-4132-a96e-abaa470a16f5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86922025-01-15T02:38:31.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29e073c4-dc2d-4957-b0d5-7104628ecb16 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86932025-01-15T02:38:31.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29e073c4-dc2d-4957-b0d5-7104628ecb16 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86942025-01-15T02:38:31.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2473 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29e073c4-dc2d-4957-b0d5-7104628ecb16 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86952025-01-15T02:38:32.798ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28cfee63-4679-4929-ae13-164366e792e6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86962025-01-15T02:38:32.798ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28cfee63-4679-4929-ae13-164366e792e6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86972025-01-15T02:38:32.800ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1956 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28cfee63-4679-4929-ae13-164366e792e6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86982025-01-15T02:38:33.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1ebbbf6-ca0c-44f5-a134-a5717dcd2518 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
86992025-01-15T02:38:33.821ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1ebbbf6-ca0c-44f5-a134-a5717dcd2518 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87002025-01-15T02:38:33.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2101 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c1ebbbf6-ca0c-44f5-a134-a5717dcd2518 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87012025-01-15T02:38:34.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 096b4c87-92c6-4f87-a3d4-e729da6b8e42 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87022025-01-15T02:38:34.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 096b4c87-92c6-4f87-a3d4-e729da6b8e42 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87032025-01-15T02:38:34.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2505 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 096b4c87-92c6-4f87-a3d4-e729da6b8e42 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87042025-01-15T02:38:35.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 605 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 602e95e9-9afd-4ca0-96ff-760daa708e8c response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
87052025-01-15T02:38:35.866ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c6cdebae-2092-49ae-9c58-15865d06c61d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87062025-01-15T02:38:35.867ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c6cdebae-2092-49ae-9c58-15865d06c61d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87072025-01-15T02:38:35.868ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2268 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c6cdebae-2092-49ae-9c58-15865d06c61d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87082025-01-15T02:38:36.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c8592364-0f4c-41d1-ab33-93d40acc4448 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87092025-01-15T02:38:36.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c8592364-0f4c-41d1-ab33-93d40acc4448 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87102025-01-15T02:38:36.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2490 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c8592364-0f4c-41d1-ab33-93d40acc4448 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87112025-01-15T02:38:37.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 238bb042-b4b9-45e2-ad43-7011cd1f16b0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87122025-01-15T02:38:37.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 238bb042-b4b9-45e2-ad43-7011cd1f16b0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87132025-01-15T02:38:37.916ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2706 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 238bb042-b4b9-45e2-ad43-7011cd1f16b0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87142025-01-15T02:38:38.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 942ec50b-ad2f-47db-a549-5114eecf9647 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87152025-01-15T02:38:38.937ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 942ec50b-ad2f-47db-a549-5114eecf9647 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87162025-01-15T02:38:38.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2584 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 942ec50b-ad2f-47db-a549-5114eecf9647 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87172025-01-15T02:38:39.959ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3b9a540-c554-49cf-a283-5c776835c760 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87182025-01-15T02:38:39.960ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3b9a540-c554-49cf-a283-5c776835c760 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87192025-01-15T02:38:39.961ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1996 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a3b9a540-c554-49cf-a283-5c776835c760 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
87202025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
87212025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
87222025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
87232025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
87242025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
87252025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
87262025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
87272025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
87282025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
87292025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
87302025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87312025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
87322025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87332025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
87342025-01-15T02:38:40.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
87352025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87362025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
87372025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
87382025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87392025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
87402025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
87412025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87422025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
87432025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87442025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
87452025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
87462025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
87472025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
87482025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
87492025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
87502025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
87512025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87522025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
87532025-01-15T02:38:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
87542025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87552025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87562025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
87572025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
87582025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87592025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
87602025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
87612025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
87622025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
87632025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
87642025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
87652025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
87662025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
87672025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
87682025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
87692025-01-15T02:38:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
87702025-01-15T02:38:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
87712025-01-15T02:38:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
87722025-01-15T02:38:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
87732025-01-15T02:38:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
87742025-01-15T02:38:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4007, tv_nsec: 344341896 } })
87752025-01-15T02:38:40.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
87762025-01-15T02:38:40.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4007, tv_nsec: 344817419 } })
87772025-01-15T02:38:40.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
87782025-01-15T02:38:40.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87792025-01-15T02:38:40.506ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.931516ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
87802025-01-15T02:38:40.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87812025-01-15T02:38:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87822025-01-15T02:38:40.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87832025-01-15T02:38:40.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
87842025-01-15T02:38:40.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87852025-01-15T02:38:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87862025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
87872025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
87882025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
87892025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
87902025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87912025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
87922025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87932025-01-15T02:38:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
87942025-01-15T02:38:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87952025-01-15T02:38:40.511ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87962025-01-15T02:38:40.513ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87972025-01-15T02:38:40.513ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87982025-01-15T02:38:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
87992025-01-15T02:38:40.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
88002025-01-15T02:38:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
88012025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.842606ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
88022025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
88032025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88042025-01-15T02:38:40.514ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
88052025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
88062025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
88072025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
88082025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
88092025-01-15T02:38:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88102025-01-15T02:38:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
88112025-01-15T02:38:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
88122025-01-15T02:38:40.516ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88132025-01-15T02:38:40.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88142025-01-15T02:38:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88152025-01-15T02:38:40.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
88162025-01-15T02:38:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
88172025-01-15T02:38:40.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
88182025-01-15T02:38:40.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88192025-01-15T02:38:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88202025-01-15T02:38:40.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88212025-01-15T02:38:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88222025-01-15T02:38:40.519ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
88232025-01-15T02:38:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
88242025-01-15T02:38:40.520ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
88252025-01-15T02:38:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
88262025-01-15T02:38:40.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4007, tv_nsec: 359773963 } })
88272025-01-15T02:38:40.521ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
88282025-01-15T02:38:40.522ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
88292025-01-15T02:38:40.522ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
88302025-01-15T02:38:40.522ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
88312025-01-15T02:38:40.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
88322025-01-15T02:38:40.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88332025-01-15T02:38:40.523ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88342025-01-15T02:38:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
88352025-01-15T02:38:40.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
88362025-01-15T02:38:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
88372025-01-15T02:38:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
88382025-01-15T02:38:40.525ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
88392025-01-15T02:38:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
88402025-01-15T02:38:40.541ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88412025-01-15T02:38:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88422025-01-15T02:38:40.552ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
88432025-01-15T02:38:40.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
88442025-01-15T02:38:40.554ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 50.503363ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
88452025-01-15T02:38:40.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88462025-01-15T02:38:40.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
88472025-01-15T02:38:40.560ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88482025-01-15T02:38:40.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88492025-01-15T02:38:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88502025-01-15T02:38:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88512025-01-15T02:38:40.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 57.447921ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
88522025-01-15T02:38:40.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88532025-01-15T02:38:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88542025-01-15T02:38:40.566ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
88552025-01-15T02:38:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
88562025-01-15T02:38:40.568ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
88572025-01-15T02:38:40.568ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
88582025-01-15T02:38:40.568ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
88592025-01-15T02:38:40.568ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 64.673815ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
88602025-01-15T02:38:40.568ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03fe2a0c-834d-45ba-98ab-4fffeef1d666", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:39 GMT"} })
88612025-01-15T02:38:40.568ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
88622025-01-15T02:38:40.568ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
88632025-01-15T02:38:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
88642025-01-15T02:38:40.570ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
88652025-01-15T02:38:40.570ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
88662025-01-15T02:38:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 67.274946ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
88672025-01-15T02:38:40.574ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
88682025-01-15T02:38:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 71.14032ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
88692025-01-15T02:38:40.575ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
88702025-01-15T02:38:40.575ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
88712025-01-15T02:38:40.575ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "75e090ec-036f-478c-8bf4-8a13946effef", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:39 GMT"} })
88722025-01-15T02:38:40.575ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
88732025-01-15T02:38:40.576ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
88742025-01-15T02:38:40.584ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
88752025-01-15T02:38:40.584ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
88762025-01-15T02:38:40.584ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 80.680257ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
88772025-01-15T02:38:40.584ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.057107ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88782025-01-15T02:38:40.586ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
88792025-01-15T02:38:40.586ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
88802025-01-15T02:38:40.587ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "21416a91-2518-447b-bd56-53b4996b1b46", "content-length": "26", "date": "Wed, 15 Jan 2025 02:38:39 GMT"} })
88812025-01-15T02:38:40.587ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
88822025-01-15T02:38:40.589ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
88832025-01-15T02:38:40.609ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
88842025-01-15T02:38:40.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
88852025-01-15T02:38:40.611ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 106.484633ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
88862025-01-15T02:38:40.612ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88872025-01-15T02:38:40.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
88882025-01-15T02:38:40.615ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88892025-01-15T02:38:40.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88902025-01-15T02:38:40.640ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 136.050241ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
88912025-01-15T02:38:40.645ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
88922025-01-15T02:38:40.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
88932025-01-15T02:38:40.647ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 144.232482ms iteration = 27 status = {"physical_disks_added":0}
88942025-01-15T02:38:40.654ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
88952025-01-15T02:38:40.655ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "07efbbac-59ac-434b-9dec-9b6b159dce7e", "content-length": "2", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
88962025-01-15T02:38:40.656ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
88972025-01-15T02:38:40.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
88982025-01-15T02:38:40.667ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 164.287174ms iteration = 24 status = {}
88992025-01-15T02:38:40.677ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
89002025-01-15T02:38:40.677ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
89012025-01-15T02:38:40.677ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eb0af68-0da1-49d2-8b6b-12496a0e67d8", "content-length": "372", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89022025-01-15T02:38:40.677ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89032025-01-15T02:38:40.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89042025-01-15T02:38:40.679ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 175.522156ms iteration = 24 status = 0
89052025-01-15T02:38:40.680ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 177.52591ms iteration = 24 status = null
89062025-01-15T02:38:40.705ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89072025-01-15T02:38:40.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
89082025-01-15T02:38:40.707ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89092025-01-15T02:38:40.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89102025-01-15T02:38:40.719ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
89112025-01-15T02:38:40.719ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
89122025-01-15T02:38:40.729ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89132025-01-15T02:38:40.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89142025-01-15T02:38:40.732ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
89152025-01-15T02:38:40.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89162025-01-15T02:38:40.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89172025-01-15T02:38:40.735ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89182025-01-15T02:38:40.737ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
89192025-01-15T02:38:40.737ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89202025-01-15T02:38:40.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89212025-01-15T02:38:40.741ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
89222025-01-15T02:38:40.742ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "87472f89-f94e-4c59-918a-f3bf7fd4a944", "content-length": "134", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89232025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
89242025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
89252025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
89262025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
89272025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89282025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89292025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89302025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
89312025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89322025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
89332025-01-15T02:38:40.742ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
89342025-01-15T02:38:40.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89352025-01-15T02:38:40.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89362025-01-15T02:38:40.745ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
89372025-01-15T02:38:40.745ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa934d0f-3a0b-4602-a624-eea91a0f5263", "content-length": "2", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89382025-01-15T02:38:40.745ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
89392025-01-15T02:38:40.745ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f54d658-d550-4342-9990-2d04f031561a", "content-length": "52", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89402025-01-15T02:38:40.745ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
89412025-01-15T02:38:40.745ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
89422025-01-15T02:38:40.802ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f9ac201e-e5f1-464e-983c-000ebcf0340e", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89432025-01-15T02:38:40.802ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89442025-01-15T02:38:40.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89452025-01-15T02:38:40.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89462025-01-15T02:38:40.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89472025-01-15T02:38:40.824ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
89482025-01-15T02:38:40.824ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9226e226-b168-4964-bd19-7516a1b4963d", "content-length": "691", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89492025-01-15T02:38:40.824ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
89502025-01-15T02:38:40.824ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
89512025-01-15T02:38:40.824ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
89522025-01-15T02:38:40.825ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 321.621329ms iteration = 25 status = {}
89532025-01-15T02:38:40.827ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89542025-01-15T02:38:40.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89552025-01-15T02:38:40.830ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
89562025-01-15T02:38:40.830ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
89572025-01-15T02:38:40.834ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a386b288-fc45-479a-bd3d-c7fad63274cf", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89582025-01-15T02:38:40.836ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 332.522613ms iteration = 25 status = {}
89592025-01-15T02:38:40.905ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "795e4932-e9e5-42ed-b1b4-9819052484fc", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89602025-01-15T02:38:40.905ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
89612025-01-15T02:38:40.905ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
89622025-01-15T02:38:40.905ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "86cb0dd2-bd47-4c0a-9b79-6d2b3565007e", "content-length": "682", "date": "Wed, 15 Jan 2025 02:38:40 GMT"} })
89632025-01-15T02:38:40.905ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89642025-01-15T02:38:40.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89652025-01-15T02:38:40.909ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89662025-01-15T02:38:40.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89672025-01-15T02:38:40.913ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89682025-01-15T02:38:40.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89692025-01-15T02:38:40.915ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89702025-01-15T02:38:40.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89712025-01-15T02:38:40.921ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 417.958408ms iteration = 24 status = {}
89722025-01-15T02:38:40.981ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fed847f2-8e85-4ddc-b6e9-b9ccebc875ff roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89732025-01-15T02:38:40.982ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fed847f2-8e85-4ddc-b6e9-b9ccebc875ff resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89742025-01-15T02:38:40.984ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2528 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fed847f2-8e85-4ddc-b6e9-b9ccebc875ff response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89752025-01-15T02:38:42.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d62a3bbd-e443-4d32-bbd1-c6476dba3005 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89762025-01-15T02:38:42.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d62a3bbd-e443-4d32-bbd1-c6476dba3005 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89772025-01-15T02:38:42.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2496 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d62a3bbd-e443-4d32-bbd1-c6476dba3005 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89782025-01-15T02:38:43.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 452fb362-a7f8-4ae7-8f4f-db067a180f78 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89792025-01-15T02:38:43.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 452fb362-a7f8-4ae7-8f4f-db067a180f78 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89802025-01-15T02:38:43.027ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2629 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 452fb362-a7f8-4ae7-8f4f-db067a180f78 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89812025-01-15T02:38:44.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6eeb0eec-378e-415e-9def-fa8356f690d5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89822025-01-15T02:38:44.047ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6eeb0eec-378e-415e-9def-fa8356f690d5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89832025-01-15T02:38:44.049ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2446 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6eeb0eec-378e-415e-9def-fa8356f690d5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89842025-01-15T02:38:45.067ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 473ea47a-149f-4996-a3be-a45119ac4021 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89852025-01-15T02:38:45.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 473ea47a-149f-4996-a3be-a45119ac4021 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89862025-01-15T02:38:45.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2140 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 473ea47a-149f-4996-a3be-a45119ac4021 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89872025-01-15T02:38:45.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 556 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = a21e2aba-6ab1-4eee-9a4e-8052a8c80b94 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
89882025-01-15T02:38:46.089ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 388c0890-546d-4b32-b700-84be1b87cc40 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89892025-01-15T02:38:46.089ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 388c0890-546d-4b32-b700-84be1b87cc40 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89902025-01-15T02:38:46.091ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2755 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 388c0890-546d-4b32-b700-84be1b87cc40 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89912025-01-15T02:38:47.111ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d984d83c-5856-4e0a-be3f-e804113e4c6a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89922025-01-15T02:38:47.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d984d83c-5856-4e0a-be3f-e804113e4c6a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89932025-01-15T02:38:47.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d984d83c-5856-4e0a-be3f-e804113e4c6a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89942025-01-15T02:38:48.132ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cb90be9a-c74d-46d2-8bb2-c63f5a61b598 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89952025-01-15T02:38:48.132ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cb90be9a-c74d-46d2-8bb2-c63f5a61b598 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89962025-01-15T02:38:48.133ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1950 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cb90be9a-c74d-46d2-8bb2-c63f5a61b598 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89972025-01-15T02:38:49.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 27fce7da-ecfb-4c20-b2d1-5c449c9ed871 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89982025-01-15T02:38:49.154ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 27fce7da-ecfb-4c20-b2d1-5c449c9ed871 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
89992025-01-15T02:38:49.156ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 27fce7da-ecfb-4c20-b2d1-5c449c9ed871 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90002025-01-15T02:38:50.175ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c3bbff53-79d2-49e7-baf8-98fdc5d0da08 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90012025-01-15T02:38:50.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c3bbff53-79d2-49e7-baf8-98fdc5d0da08 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90022025-01-15T02:38:50.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = c3bbff53-79d2-49e7-baf8-98fdc5d0da08 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90032025-01-15T02:38:50.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
90042025-01-15T02:38:50.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
90052025-01-15T02:38:50.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90062025-01-15T02:38:50.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90072025-01-15T02:38:50.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90082025-01-15T02:38:50.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90092025-01-15T02:38:50.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90102025-01-15T02:38:50.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90112025-01-15T02:38:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90122025-01-15T02:38:50.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
90132025-01-15T02:38:50.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90142025-01-15T02:38:50.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90152025-01-15T02:38:50.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90162025-01-15T02:38:50.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90172025-01-15T02:38:50.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90182025-01-15T02:38:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
90192025-01-15T02:38:50.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90202025-01-15T02:38:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90212025-01-15T02:38:50.551ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
90222025-01-15T02:38:50.551ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.534943ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
90232025-01-15T02:38:50.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.55236ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
90242025-01-15T02:38:51.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8141beef-2d0c-451b-8bde-3ccf6e625d89 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90252025-01-15T02:38:51.197ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8141beef-2d0c-451b-8bde-3ccf6e625d89 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90262025-01-15T02:38:51.199ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2251 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8141beef-2d0c-451b-8bde-3ccf6e625d89 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90272025-01-15T02:38:52.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 057d8dfe-6be5-4f00-a7c7-29af3b9003db roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90282025-01-15T02:38:52.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 057d8dfe-6be5-4f00-a7c7-29af3b9003db resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90292025-01-15T02:38:52.221ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2094 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 057d8dfe-6be5-4f00-a7c7-29af3b9003db response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90302025-01-15T02:38:53.240ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cbdbffba-184f-484a-b632-396cd2014b84 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90312025-01-15T02:38:53.240ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cbdbffba-184f-484a-b632-396cd2014b84 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90322025-01-15T02:38:53.242ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2269 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cbdbffba-184f-484a-b632-396cd2014b84 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90332025-01-15T02:38:54.260ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 198dd237-21a3-46e9-ae79-cff119485345 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90342025-01-15T02:38:54.260ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 198dd237-21a3-46e9-ae79-cff119485345 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90352025-01-15T02:38:54.262ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2377 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 198dd237-21a3-46e9-ae79-cff119485345 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90362025-01-15T02:38:55.282ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63c0b8df-e1cd-46c6-b3d6-58bd8fd737d4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90372025-01-15T02:38:55.282ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63c0b8df-e1cd-46c6-b3d6-58bd8fd737d4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90382025-01-15T02:38:55.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63c0b8df-e1cd-46c6-b3d6-58bd8fd737d4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90392025-01-15T02:38:55.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 773 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = eb0f514d-d9c3-4c0c-bd6a-8fdde6593463 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
90402025-01-15T02:38:56.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f042ea5f-7092-4f8d-9683-9ae0b3836e98 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90412025-01-15T02:38:56.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f042ea5f-7092-4f8d-9683-9ae0b3836e98 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90422025-01-15T02:38:56.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2094 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f042ea5f-7092-4f8d-9683-9ae0b3836e98 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90432025-01-15T02:38:57.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9215775b-4737-4d50-b98d-cc7af8d814e6 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90442025-01-15T02:38:57.327ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9215775b-4737-4d50-b98d-cc7af8d814e6 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90452025-01-15T02:38:57.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2506 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9215775b-4737-4d50-b98d-cc7af8d814e6 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90462025-01-15T02:38:58.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a46954be-d91a-436e-b072-9735aee33032 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90472025-01-15T02:38:58.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a46954be-d91a-436e-b072-9735aee33032 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90482025-01-15T02:38:58.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2312 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a46954be-d91a-436e-b072-9735aee33032 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90492025-01-15T02:38:59.370ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0b500c1f-1910-4970-86cf-ab84d1e71c4b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90502025-01-15T02:38:59.370ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0b500c1f-1910-4970-86cf-ab84d1e71c4b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90512025-01-15T02:38:59.372ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2371 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0b500c1f-1910-4970-86cf-ab84d1e71c4b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90522025-01-15T02:39:00.392ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5060189f-4158-4300-baac-ce72a633bb7b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90532025-01-15T02:39:00.393ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5060189f-4158-4300-baac-ce72a633bb7b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90542025-01-15T02:39:00.395ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2322 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5060189f-4158-4300-baac-ce72a633bb7b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90552025-01-15T02:39:00.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
90562025-01-15T02:39:00.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
90572025-01-15T02:39:00.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90582025-01-15T02:39:00.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90592025-01-15T02:39:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90602025-01-15T02:39:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90612025-01-15T02:39:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90622025-01-15T02:39:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
90632025-01-15T02:39:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90642025-01-15T02:39:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90652025-01-15T02:39:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90662025-01-15T02:39:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90672025-01-15T02:39:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90682025-01-15T02:39:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
90692025-01-15T02:39:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90702025-01-15T02:39:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90712025-01-15T02:39:00.522ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
90722025-01-15T02:39:00.522ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.169187ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
90732025-01-15T02:39:00.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90742025-01-15T02:39:00.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90752025-01-15T02:39:00.559ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.775666ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
90762025-01-15T02:39:01.414ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fa388541-2a04-43db-82c5-77e7713447be roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90772025-01-15T02:39:01.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fa388541-2a04-43db-82c5-77e7713447be resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90782025-01-15T02:39:01.417ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2523 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = fa388541-2a04-43db-82c5-77e7713447be response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90792025-01-15T02:39:02.437ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4c8b0386-9bb4-421a-964c-4cf45f3fc925 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90802025-01-15T02:39:02.437ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4c8b0386-9bb4-421a-964c-4cf45f3fc925 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90812025-01-15T02:39:02.439ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2234 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4c8b0386-9bb4-421a-964c-4cf45f3fc925 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90822025-01-15T02:39:03.459ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 578aa5a7-86c6-4982-b73f-4f8a41d0d600 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90832025-01-15T02:39:03.460ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 578aa5a7-86c6-4982-b73f-4f8a41d0d600 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90842025-01-15T02:39:03.462ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2383 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 578aa5a7-86c6-4982-b73f-4f8a41d0d600 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90852025-01-15T02:39:04.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2365371e-f52f-4fc0-9eb1-0d0e1b6e5e47 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90862025-01-15T02:39:04.483ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2365371e-f52f-4fc0-9eb1-0d0e1b6e5e47 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90872025-01-15T02:39:04.484ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1880 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2365371e-f52f-4fc0-9eb1-0d0e1b6e5e47 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90882025-01-15T02:39:05.504ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82aef22e-36fb-41a6-b07f-9703caca8f6b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90892025-01-15T02:39:05.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82aef22e-36fb-41a6-b07f-9703caca8f6b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90902025-01-15T02:39:05.506ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2535 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 82aef22e-36fb-41a6-b07f-9703caca8f6b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90912025-01-15T02:39:05.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 540 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 96fb6c1d-86f1-40ae-90c7-48848e8783d8 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
90922025-01-15T02:39:06.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7ec7ae14-e967-4ab4-b8ba-c28635a95473 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90932025-01-15T02:39:06.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7ec7ae14-e967-4ab4-b8ba-c28635a95473 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90942025-01-15T02:39:06.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2094 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7ec7ae14-e967-4ab4-b8ba-c28635a95473 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90952025-01-15T02:39:07.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 809c2257-564a-45ed-badb-89ac5e6e7533 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90962025-01-15T02:39:07.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 809c2257-564a-45ed-badb-89ac5e6e7533 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90972025-01-15T02:39:07.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2162 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 809c2257-564a-45ed-badb-89ac5e6e7533 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90982025-01-15T02:39:08.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46701508-003e-42a1-8723-3c22fdf82b1e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
90992025-01-15T02:39:08.571ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46701508-003e-42a1-8723-3c22fdf82b1e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
91002025-01-15T02:39:08.573ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2456 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 46701508-003e-42a1-8723-3c22fdf82b1e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
91012025-01-15T02:39:09.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 16625e84-24f4-44aa-8693-71df1127c9cf roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
91022025-01-15T02:39:09.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 16625e84-24f4-44aa-8693-71df1127c9cf resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
91032025-01-15T02:39:09.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1880 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 16625e84-24f4-44aa-8693-71df1127c9cf response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
91042025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
91052025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
91062025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
91072025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
91082025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
91092025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
91102025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
91112025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
91122025-01-15T02:39:10.502ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:29:10.502247733 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
91132025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
91142025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
91152025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
91162025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
91172025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
91182025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
91192025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
91202025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
91212025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
91222025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
91232025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
91242025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
91252025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
91262025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
91272025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
91282025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
91292025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
91302025-01-15T02:39:10.502ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 4037, tv_nsec: 341388797 } })
91312025-01-15T02:39:10.502ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 4037, tv_nsec: 341428868 } })
91322025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
91332025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
91342025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91352025-01-15T02:39:10.502ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
91362025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
91372025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
91382025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
91392025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91402025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
91412025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
91422025-01-15T02:39:10.503ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
91432025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
91442025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
91452025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
91462025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91472025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
91482025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91492025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
91502025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
91512025-01-15T02:39:10.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
91522025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91532025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
91542025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
91552025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
91562025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
91572025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
91582025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
91592025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
91602025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
91612025-01-15T02:39:10.503ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
91622025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
91632025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.581\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
91642025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
91652025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91662025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
91672025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91682025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91692025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
91702025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
91712025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
91722025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.273554ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
91732025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
91742025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
91752025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
91762025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
91772025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.366957ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
91782025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
91792025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91802025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
91812025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
91822025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91832025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91842025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91852025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
91862025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
91872025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
91882025-01-15T02:39:10.503ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
91892025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
91902025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
91912025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
91922025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
91932025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
91942025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
91952025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
91962025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
91972025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
91982025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
91992025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
92002025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
92012025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
92022025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
92032025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
92042025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
92052025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
92062025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
92072025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
92082025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
92092025-01-15T02:39:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
92102025-01-15T02:39:10.505ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4037, tv_nsec: 344229873 } })
92112025-01-15T02:39:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
92122025-01-15T02:39:10.505ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
92132025-01-15T02:39:10.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4037, tv_nsec: 344657915 } })
92142025-01-15T02:39:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92152025-01-15T02:39:10.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
92162025-01-15T02:39:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92172025-01-15T02:39:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92182025-01-15T02:39:10.506ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:29:10.502247733 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
92192025-01-15T02:39:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.547863ms iteration = 13 status = {"expiration":"2025-01-15T02:29:10.502247733Z","pruned":[]}
92202025-01-15T02:39:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.649585ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
92212025-01-15T02:39:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92222025-01-15T02:39:10.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
92232025-01-15T02:39:10.508ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
92242025-01-15T02:39:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
92252025-01-15T02:39:10.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92262025-01-15T02:39:10.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92272025-01-15T02:39:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
92282025-01-15T02:39:10.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92292025-01-15T02:39:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92302025-01-15T02:39:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92312025-01-15T02:39:10.513ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92322025-01-15T02:39:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92332025-01-15T02:39:10.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92342025-01-15T02:39:10.522ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92352025-01-15T02:39:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
92362025-01-15T02:39:10.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
92372025-01-15T02:39:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92382025-01-15T02:39:10.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
92392025-01-15T02:39:10.524ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
92402025-01-15T02:39:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
92412025-01-15T02:39:10.549ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92422025-01-15T02:39:10.551ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92432025-01-15T02:39:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
92442025-01-15T02:39:10.551ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
92452025-01-15T02:39:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
92462025-01-15T02:39:10.553ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92472025-01-15T02:39:10.554ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 49.995249ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
92482025-01-15T02:39:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
92492025-01-15T02:39:10.554ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
92502025-01-15T02:39:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
92512025-01-15T02:39:10.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92522025-01-15T02:39:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
92532025-01-15T02:39:10.555ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
92542025-01-15T02:39:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
92552025-01-15T02:39:10.556ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92562025-01-15T02:39:10.557ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
92572025-01-15T02:39:10.557ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
92582025-01-15T02:39:10.557ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
92592025-01-15T02:39:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
92602025-01-15T02:39:10.557ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
92612025-01-15T02:39:10.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92622025-01-15T02:39:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
92632025-01-15T02:39:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92642025-01-15T02:39:10.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92652025-01-15T02:39:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92662025-01-15T02:39:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
92672025-01-15T02:39:10.559ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
92682025-01-15T02:39:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
92692025-01-15T02:39:10.560ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
92702025-01-15T02:39:10.560ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92712025-01-15T02:39:10.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92722025-01-15T02:39:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92732025-01-15T02:39:10.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92742025-01-15T02:39:10.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
92752025-01-15T02:39:10.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92762025-01-15T02:39:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
92772025-01-15T02:39:10.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92782025-01-15T02:39:10.562ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
92792025-01-15T02:39:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
92802025-01-15T02:39:10.563ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4037, tv_nsec: 401627802 } })
92812025-01-15T02:39:10.563ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
92822025-01-15T02:39:10.563ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
92832025-01-15T02:39:10.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
92842025-01-15T02:39:10.563ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
92852025-01-15T02:39:10.563ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92862025-01-15T02:39:10.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92872025-01-15T02:39:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92882025-01-15T02:39:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
92892025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
92902025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
92912025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
92922025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92932025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
92942025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
92952025-01-15T02:39:10.565ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
92962025-01-15T02:39:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92972025-01-15T02:39:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
92982025-01-15T02:39:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92992025-01-15T02:39:10.566ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93002025-01-15T02:39:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
93012025-01-15T02:39:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
93022025-01-15T02:39:10.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93032025-01-15T02:39:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93042025-01-15T02:39:10.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93052025-01-15T02:39:10.573ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
93062025-01-15T02:39:10.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93072025-01-15T02:39:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93082025-01-15T02:39:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 70.108412ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
93092025-01-15T02:39:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 70.098761ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
93102025-01-15T02:39:10.578ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93112025-01-15T02:39:10.582ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.82989ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
93122025-01-15T02:39:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
93132025-01-15T02:39:10.583ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93142025-01-15T02:39:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93152025-01-15T02:39:10.584ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
93162025-01-15T02:39:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
93172025-01-15T02:39:10.585ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93182025-01-15T02:39:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 83.192235ms iteration = 6 status = {"nsuccess":5}
93192025-01-15T02:39:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
93202025-01-15T02:39:10.588ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93212025-01-15T02:39:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93222025-01-15T02:39:10.590ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
93232025-01-15T02:39:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
93242025-01-15T02:39:10.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93252025-01-15T02:39:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
93262025-01-15T02:39:10.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93272025-01-15T02:39:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93282025-01-15T02:39:10.598ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
93292025-01-15T02:39:10.598ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
93302025-01-15T02:39:10.598ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 94.359646ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
93312025-01-15T02:39:10.614ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
93322025-01-15T02:39:10.614ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
93332025-01-15T02:39:10.614ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
93342025-01-15T02:39:10.614ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
93352025-01-15T02:39:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
93362025-01-15T02:39:10.614ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a0b1517-0ce1-46d4-b495-ff3eb54a7666 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
93372025-01-15T02:39:10.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a0b1517-0ce1-46d4-b495-ff3eb54a7666 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
93382025-01-15T02:39:10.616ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93392025-01-15T02:39:10.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3178 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2a0b1517-0ce1-46d4-b495-ff3eb54a7666 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
93402025-01-15T02:39:10.618ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
93412025-01-15T02:39:10.618ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 115.24583ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
93422025-01-15T02:39:10.619ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93432025-01-15T02:39:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93442025-01-15T02:39:10.620ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
93452025-01-15T02:39:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
93462025-01-15T02:39:10.621ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93472025-01-15T02:39:10.621ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
93482025-01-15T02:39:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
93492025-01-15T02:39:10.621ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
93502025-01-15T02:39:10.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
93512025-01-15T02:39:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
93522025-01-15T02:39:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
93532025-01-15T02:39:10.624ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93542025-01-15T02:39:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93552025-01-15T02:39:10.627ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
93562025-01-15T02:39:10.627ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
93572025-01-15T02:39:10.627ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
93582025-01-15T02:39:10.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93592025-01-15T02:39:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
93602025-01-15T02:39:10.627ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
93612025-01-15T02:39:10.628ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 125.159527ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
93622025-01-15T02:39:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
93632025-01-15T02:39:10.628ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93642025-01-15T02:39:10.629ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 124.552321ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
93652025-01-15T02:39:10.630ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
93662025-01-15T02:39:10.632ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
93672025-01-15T02:39:10.632ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
93682025-01-15T02:39:10.632ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 129.414532ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
93692025-01-15T02:39:10.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
93702025-01-15T02:39:10.632ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93712025-01-15T02:39:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
93722025-01-15T02:39:10.634ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
93732025-01-15T02:39:10.634ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 131.491408ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
93742025-01-15T02:39:10.638ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
93752025-01-15T02:39:10.638ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93762025-01-15T02:39:10.639ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eac7e0d-d1e7-437c-9828-c67685be7097", "content-length": "26", "date": "Wed, 15 Jan 2025 02:39:09 GMT"} })
93772025-01-15T02:39:10.639ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93782025-01-15T02:39:10.639ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93792025-01-15T02:39:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93802025-01-15T02:39:10.641ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93812025-01-15T02:39:10.641ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
93822025-01-15T02:39:10.642ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
93832025-01-15T02:39:10.642ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93842025-01-15T02:39:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93852025-01-15T02:39:10.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93862025-01-15T02:39:10.659ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 154.480339ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
93872025-01-15T02:39:10.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
93882025-01-15T02:39:10.659ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93892025-01-15T02:39:10.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93902025-01-15T02:39:10.662ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 159.011701ms iteration = 25 status = 0
93912025-01-15T02:39:10.662ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
93922025-01-15T02:39:10.662ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
93932025-01-15T02:39:10.662ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93942025-01-15T02:39:10.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
93952025-01-15T02:39:10.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93962025-01-15T02:39:10.664ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93972025-01-15T02:39:10.664ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
93982025-01-15T02:39:10.664ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 161.930939ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
93992025-01-15T02:39:10.665ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
94002025-01-15T02:39:10.665ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
94012025-01-15T02:39:10.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
94022025-01-15T02:39:10.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
94032025-01-15T02:39:10.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94042025-01-15T02:39:10.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
94052025-01-15T02:39:10.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
94062025-01-15T02:39:10.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
94072025-01-15T02:39:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
94082025-01-15T02:39:10.670ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
94092025-01-15T02:39:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
94102025-01-15T02:39:10.671ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
94112025-01-15T02:39:10.671ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
94122025-01-15T02:39:10.671ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
94132025-01-15T02:39:10.671ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 168.97555ms iteration = 13 status = {"generation":1}
94142025-01-15T02:39:10.672ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
94152025-01-15T02:39:10.672ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
94162025-01-15T02:39:10.672ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
94172025-01-15T02:39:10.672ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 170.548093ms iteration = 13 status = {"generation":2}
94182025-01-15T02:39:10.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94192025-01-15T02:39:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
94202025-01-15T02:39:10.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
94212025-01-15T02:39:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
94222025-01-15T02:39:10.687ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
94232025-01-15T02:39:10.687ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 185.238589ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
94242025-01-15T02:39:10.708ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
94252025-01-15T02:39:10.708ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 205.381743ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
94262025-01-15T02:39:10.715ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94272025-01-15T02:39:10.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94282025-01-15T02:39:10.718ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 215.519506ms iteration = 28 status = {"physical_disks_added":0}
94292025-01-15T02:39:10.720ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
94302025-01-15T02:39:10.720ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 217.102978ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
94312025-01-15T02:39:10.741ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
94322025-01-15T02:39:10.741ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
94332025-01-15T02:39:10.741ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f3d7cae-7fbd-4592-a63d-f75491f0ff03", "content-length": "26", "date": "Wed, 15 Jan 2025 02:39:09 GMT"} })
94342025-01-15T02:39:10.741ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
94352025-01-15T02:39:10.743ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
94362025-01-15T02:39:10.759ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
94372025-01-15T02:39:10.794ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
94382025-01-15T02:39:10.794ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
94392025-01-15T02:39:10.794ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "390d4a6c-de7e-4e2b-8ac9-a4175af8647a", "content-length": "26", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
94402025-01-15T02:39:10.795ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
94412025-01-15T02:39:10.797ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
94422025-01-15T02:39:10.808ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
94432025-01-15T02:39:10.853ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
94442025-01-15T02:39:10.853ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bf3d57f-a0b3-4c7a-97a4-65f75dde33db", "content-length": "2", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
94452025-01-15T02:39:10.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
94462025-01-15T02:39:10.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
94472025-01-15T02:39:10.861ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
94482025-01-15T02:39:10.861ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
94492025-01-15T02:39:10.862ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eaf265b-69a5-46f4-ae15-cb49fa14c746", "content-length": "372", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
94502025-01-15T02:39:10.862ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94512025-01-15T02:39:10.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94522025-01-15T02:39:10.863ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94532025-01-15T02:39:10.864ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 362.02846ms iteration = 25 status = null
94542025-01-15T02:39:10.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
94552025-01-15T02:39:10.865ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94562025-01-15T02:39:10.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94572025-01-15T02:39:10.866ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94582025-01-15T02:39:10.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
94592025-01-15T02:39:10.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
94602025-01-15T02:39:10.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94612025-01-15T02:39:10.892ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 389.897111ms iteration = 25 status = {}
94622025-01-15T02:39:10.906ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
94632025-01-15T02:39:10.906ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
94642025-01-15T02:39:10.906ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4c6ff6d5-d399-4699-ad5a-7c6272199f37", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
94652025-01-15T02:39:10.906ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94662025-01-15T02:39:10.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94672025-01-15T02:39:10.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94682025-01-15T02:39:10.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94692025-01-15T02:39:10.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94702025-01-15T02:39:10.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94712025-01-15T02:39:10.914ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
94722025-01-15T02:39:10.915ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "95b9026b-49c2-4f26-84fc-3460671e1e15", "date": "Wed, 15 Jan 2025 02:39:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94732025-01-15T02:39:10.915ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94742025-01-15T02:39:10.916ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "17d2850e-2a70-46cb-af00-3c8cb691740d", "date": "Wed, 15 Jan 2025 02:39:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94752025-01-15T02:39:10.916ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94762025-01-15T02:39:10.918ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
94772025-01-15T02:39:10.918ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 416.100708ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
94782025-01-15T02:39:10.919ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
94792025-01-15T02:39:10.919ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f558cd00-0e6b-4058-b9a0-9ec8694f2b25", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94802025-01-15T02:39:10.919ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94812025-01-15T02:39:10.920ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "f947ed4c-c335-4ea4-919d-0a7d71dbb2ff", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94822025-01-15T02:39:10.920ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94832025-01-15T02:39:10.922ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "897ea8eb-1a6f-49ec-bb33-d70cd7adcbc0", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94842025-01-15T02:39:10.922ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94852025-01-15T02:39:10.924ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
94862025-01-15T02:39:10.924ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 421.848774ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
94872025-01-15T02:39:10.936ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
94882025-01-15T02:39:10.939ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
94892025-01-15T02:39:10.939ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
94902025-01-15T02:39:10.981ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94912025-01-15T02:39:10.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94922025-01-15T02:39:10.983ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
94932025-01-15T02:39:10.983ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94942025-01-15T02:39:10.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94952025-01-15T02:39:10.994ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
94962025-01-15T02:39:10.994ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "96e2438b-f1b7-490d-baa0-2dd2696e4901", "content-length": "134", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
94972025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
94982025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
94992025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
95002025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
95012025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95022025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95032025-01-15T02:39:10.994ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95042025-01-15T02:39:10.995ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
95052025-01-15T02:39:10.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95062025-01-15T02:39:10.995ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
95072025-01-15T02:39:10.995ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
95082025-01-15T02:39:10.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95092025-01-15T02:39:10.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95102025-01-15T02:39:10.997ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
95112025-01-15T02:39:10.997ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "110f3e0b-f7eb-4284-9208-74e6c27eeff2", "content-length": "2", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
95122025-01-15T02:39:10.997ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
95132025-01-15T02:39:10.997ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc4de9f7-8d34-42b7-ba95-cf11173517d8", "content-length": "52", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
95142025-01-15T02:39:10.997ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
95152025-01-15T02:39:10.997ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
95162025-01-15T02:39:11.019ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ce4a1af4-3bdf-4220-ab0d-ccb6d4a86feb", "date": "Wed, 15 Jan 2025 02:39:10 GMT"} })
95172025-01-15T02:39:11.020ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 517.489015ms iteration = 26 status = {}
95182025-01-15T02:39:11.020ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f72c1f17-e34b-4ac0-a153-169ddd7a6bc6", "content-length": "691", "date": "Wed, 15 Jan 2025 02:39:11 GMT"} })
95192025-01-15T02:39:11.020ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
95202025-01-15T02:39:11.020ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
95212025-01-15T02:39:11.020ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
95222025-01-15T02:39:11.021ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 518.633915ms iteration = 26 status = {}
95232025-01-15T02:39:11.113ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cead0457-4c50-4f3e-85d4-2388218e0a9e", "date": "Wed, 15 Jan 2025 02:39:11 GMT"} })
95242025-01-15T02:39:11.113ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
95252025-01-15T02:39:11.113ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
95262025-01-15T02:39:11.113ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "eec64e35-7ce2-4366-ab80-29adb0baf611", "content-length": "682", "date": "Wed, 15 Jan 2025 02:39:11 GMT"} })
95272025-01-15T02:39:11.113ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95282025-01-15T02:39:11.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95292025-01-15T02:39:11.118ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95302025-01-15T02:39:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95312025-01-15T02:39:11.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95322025-01-15T02:39:11.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95332025-01-15T02:39:11.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95342025-01-15T02:39:11.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95352025-01-15T02:39:11.132ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 629.397494ms iteration = 25 status = {}
95362025-01-15T02:39:11.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b192031f-5c42-4aa1-8893-bd689944a258 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95372025-01-15T02:39:11.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b192031f-5c42-4aa1-8893-bd689944a258 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95382025-01-15T02:39:11.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2140 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b192031f-5c42-4aa1-8893-bd689944a258 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95392025-01-15T02:39:12.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 75340e8c-0cdf-4ca6-bfb7-2c9175ebbc4e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95402025-01-15T02:39:12.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 75340e8c-0cdf-4ca6-bfb7-2c9175ebbc4e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95412025-01-15T02:39:12.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2189 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 75340e8c-0cdf-4ca6-bfb7-2c9175ebbc4e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95422025-01-15T02:39:13.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 52ffcc05-c613-461d-8f67-2c49c26dbb55 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95432025-01-15T02:39:13.684ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 52ffcc05-c613-461d-8f67-2c49c26dbb55 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95442025-01-15T02:39:13.686ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2235 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 52ffcc05-c613-461d-8f67-2c49c26dbb55 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95452025-01-15T02:39:14.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 55eb3ce5-0c12-49bd-8ea2-60ca883be1d9 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95462025-01-15T02:39:14.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 55eb3ce5-0c12-49bd-8ea2-60ca883be1d9 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95472025-01-15T02:39:14.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2447 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 55eb3ce5-0c12-49bd-8ea2-60ca883be1d9 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95482025-01-15T02:39:15.728ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1cd945e4-1079-408c-97e7-8ec31b76e4c2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95492025-01-15T02:39:15.729ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1cd945e4-1079-408c-97e7-8ec31b76e4c2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95502025-01-15T02:39:15.731ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3133 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 1cd945e4-1079-408c-97e7-8ec31b76e4c2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95512025-01-15T02:39:15.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = ba20bb4e-602f-46af-8af4-71b3e468e7e3 roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
95522025-01-15T02:39:15.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = ba20bb4e-602f-46af-8af4-71b3e468e7e3 resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
95532025-01-15T02:39:15.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3348 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = ba20bb4e-602f-46af-8af4-71b3e468e7e3 response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?sort_by=id_ascending
95542025-01-15T02:39:15.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = b303003b-644e-4563-b86d-923b73ef535d roles = RoleSet { roles: {} } uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
95552025-01-15T02:39:15.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = b303003b-644e-4563-b86d-923b73ef535d resource = Database result = Ok(()) uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
95562025-01-15T02:39:15.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2191 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48700 req_id = b303003b-644e-4563-b86d-923b73ef535d response_code = 200 uri = /metrics/collectors/f71d33aa-7320-48b9-81d2-899a100e78cd/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZTc0NjczNC1lMWZmLTQwOTctOTZiMy1lMzQ0MDQxY2I2ZWMifX0%3D
95572025-01-15T02:39:15.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 503 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 172609b1-a0c8-41f7-ba78-fe650dc000e1 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
95582025-01-15T02:39:16.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63992edc-9c72-44ee-a787-d37e535285b4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95592025-01-15T02:39:16.752ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63992edc-9c72-44ee-a787-d37e535285b4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95602025-01-15T02:39:16.754ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1720 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 63992edc-9c72-44ee-a787-d37e535285b4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95612025-01-15T02:39:17.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28a072d1-0a23-4b07-b6a4-439d26fc72fa roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95622025-01-15T02:39:17.774ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28a072d1-0a23-4b07-b6a4-439d26fc72fa resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95632025-01-15T02:39:17.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 28a072d1-0a23-4b07-b6a4-439d26fc72fa response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95642025-01-15T02:39:18.796ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e711963b-c7b3-41b2-ba1f-492fa97f4181 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95652025-01-15T02:39:18.796ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e711963b-c7b3-41b2-ba1f-492fa97f4181 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95662025-01-15T02:39:18.798ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2177 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e711963b-c7b3-41b2-ba1f-492fa97f4181 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95672025-01-15T02:39:19.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 83291101-2121-4d03-a7d5-f922be335c21 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95682025-01-15T02:39:19.820ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 83291101-2121-4d03-a7d5-f922be335c21 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95692025-01-15T02:39:19.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2364 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 83291101-2121-4d03-a7d5-f922be335c21 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95702025-01-15T02:39:20.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
95712025-01-15T02:39:20.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95722025-01-15T02:39:20.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
95732025-01-15T02:39:20.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95742025-01-15T02:39:20.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95752025-01-15T02:39:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95762025-01-15T02:39:20.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95772025-01-15T02:39:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95782025-01-15T02:39:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95792025-01-15T02:39:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95802025-01-15T02:39:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95812025-01-15T02:39:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95822025-01-15T02:39:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95832025-01-15T02:39:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95842025-01-15T02:39:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95852025-01-15T02:39:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
95862025-01-15T02:39:20.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95872025-01-15T02:39:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95882025-01-15T02:39:20.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
95892025-01-15T02:39:20.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.191969ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
95902025-01-15T02:39:20.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.923145ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
95912025-01-15T02:39:20.842ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0b133a44-0faf-465e-950c-315558e04916 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95922025-01-15T02:39:20.842ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0b133a44-0faf-465e-950c-315558e04916 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95932025-01-15T02:39:20.843ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1654 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0b133a44-0faf-465e-950c-315558e04916 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95942025-01-15T02:39:21.866ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8fda28d4-4726-4751-86b8-3a343ef49dd0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95952025-01-15T02:39:21.866ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8fda28d4-4726-4751-86b8-3a343ef49dd0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95962025-01-15T02:39:21.868ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2246 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8fda28d4-4726-4751-86b8-3a343ef49dd0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95972025-01-15T02:39:22.887ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f6b0087f-f0f5-48a6-978c-abc3e5c0604e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95982025-01-15T02:39:22.888ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f6b0087f-f0f5-48a6-978c-abc3e5c0604e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
95992025-01-15T02:39:22.890ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f6b0087f-f0f5-48a6-978c-abc3e5c0604e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96002025-01-15T02:39:23.909ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e4f95a8-427e-4fb8-a6fd-5aeff63c2963 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96012025-01-15T02:39:23.909ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e4f95a8-427e-4fb8-a6fd-5aeff63c2963 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96022025-01-15T02:39:23.911ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2565 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9e4f95a8-427e-4fb8-a6fd-5aeff63c2963 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96032025-01-15T02:39:24.931ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 87aa806d-036f-4010-b23b-befa6336841f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96042025-01-15T02:39:24.931ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 87aa806d-036f-4010-b23b-befa6336841f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96052025-01-15T02:39:24.933ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 87aa806d-036f-4010-b23b-befa6336841f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96062025-01-15T02:39:25.745ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 819 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = e86cd918-4f25-4815-af8f-4cc1c4b680b1 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
96072025-01-15T02:39:25.953ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc35fd2c-3ab0-40e1-a8fc-519577a130fc roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96082025-01-15T02:39:25.954ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc35fd2c-3ab0-40e1-a8fc-519577a130fc resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96092025-01-15T02:39:25.955ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2243 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = dc35fd2c-3ab0-40e1-a8fc-519577a130fc response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96102025-01-15T02:39:26.976ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 44d0365b-8eca-405e-9a60-bf14b725660d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96112025-01-15T02:39:26.976ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 44d0365b-8eca-405e-9a60-bf14b725660d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96122025-01-15T02:39:26.978ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2423 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 44d0365b-8eca-405e-9a60-bf14b725660d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96132025-01-15T02:39:27.999ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8767b275-5b5c-4569-a771-eced998c13f2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96142025-01-15T02:39:27.999ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8767b275-5b5c-4569-a771-eced998c13f2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96152025-01-15T02:39:28.001ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2309 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8767b275-5b5c-4569-a771-eced998c13f2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96162025-01-15T02:39:29.020ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d3b1221f-b438-4b45-b576-9d64a5b79d8e roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96172025-01-15T02:39:29.021ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d3b1221f-b438-4b45-b576-9d64a5b79d8e resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96182025-01-15T02:39:29.023ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2401 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d3b1221f-b438-4b45-b576-9d64a5b79d8e response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96192025-01-15T02:39:30.043ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 397c4ef7-3a3f-4e54-b769-bf9c18dc77a4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96202025-01-15T02:39:30.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 397c4ef7-3a3f-4e54-b769-bf9c18dc77a4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96212025-01-15T02:39:30.046ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2712 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 397c4ef7-3a3f-4e54-b769-bf9c18dc77a4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96222025-01-15T02:39:30.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
96232025-01-15T02:39:30.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
96242025-01-15T02:39:30.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96252025-01-15T02:39:30.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96262025-01-15T02:39:30.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96272025-01-15T02:39:30.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96282025-01-15T02:39:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96292025-01-15T02:39:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
96302025-01-15T02:39:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96312025-01-15T02:39:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96322025-01-15T02:39:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96332025-01-15T02:39:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96342025-01-15T02:39:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96352025-01-15T02:39:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
96362025-01-15T02:39:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96372025-01-15T02:39:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96382025-01-15T02:39:30.522ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
96392025-01-15T02:39:30.522ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.26558ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
96402025-01-15T02:39:30.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96412025-01-15T02:39:30.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96422025-01-15T02:39:30.532ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.965932ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
96432025-01-15T02:39:31.065ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e0a254cd-467a-403d-8a04-d6ccd6e4aeb2 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96442025-01-15T02:39:31.066ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e0a254cd-467a-403d-8a04-d6ccd6e4aeb2 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96452025-01-15T02:39:31.068ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2406 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = e0a254cd-467a-403d-8a04-d6ccd6e4aeb2 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96462025-01-15T02:39:32.089ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 47e5f6aa-0254-4182-ad07-1cb39526b81d roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96472025-01-15T02:39:32.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 47e5f6aa-0254-4182-ad07-1cb39526b81d resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96482025-01-15T02:39:32.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2432 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 47e5f6aa-0254-4182-ad07-1cb39526b81d response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96492025-01-15T02:39:33.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 52f82678-77f6-43b2-9547-7f56c4a378ec roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96502025-01-15T02:39:33.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 52f82678-77f6-43b2-9547-7f56c4a378ec resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96512025-01-15T02:39:33.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2526 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 52f82678-77f6-43b2-9547-7f56c4a378ec response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96522025-01-15T02:39:34.132ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0f4491fc-3167-40d0-b744-fc0956af1819 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96532025-01-15T02:39:34.133ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0f4491fc-3167-40d0-b744-fc0956af1819 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96542025-01-15T02:39:34.135ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2734 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0f4491fc-3167-40d0-b744-fc0956af1819 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96552025-01-15T02:39:35.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc895847-83c7-4162-990d-8409c89a5b63 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96562025-01-15T02:39:35.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc895847-83c7-4162-990d-8409c89a5b63 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96572025-01-15T02:39:35.158ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2573 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = bc895847-83c7-4162-990d-8409c89a5b63 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96582025-01-15T02:39:35.744ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 552 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 66d8d5bc-69f3-4a65-83a1-9b2b7e71d17e response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
96592025-01-15T02:39:36.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89afbbc3-db1e-42b5-a740-37e09630f9ba roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96602025-01-15T02:39:36.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89afbbc3-db1e-42b5-a740-37e09630f9ba resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96612025-01-15T02:39:36.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2327 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89afbbc3-db1e-42b5-a740-37e09630f9ba response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96622025-01-15T02:39:37.200ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 41e656d4-d219-488e-a1ed-c63aaa3e7ab8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96632025-01-15T02:39:37.200ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 41e656d4-d219-488e-a1ed-c63aaa3e7ab8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96642025-01-15T02:39:37.202ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2494 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 41e656d4-d219-488e-a1ed-c63aaa3e7ab8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96652025-01-15T02:39:38.221ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4ba2c0ed-1b5c-47a3-8c99-7a01772c367a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96662025-01-15T02:39:38.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4ba2c0ed-1b5c-47a3-8c99-7a01772c367a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96672025-01-15T02:39:38.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2515 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4ba2c0ed-1b5c-47a3-8c99-7a01772c367a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96682025-01-15T02:39:39.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5cb51198-8d9a-4fe0-8962-a13314b82f53 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96692025-01-15T02:39:39.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5cb51198-8d9a-4fe0-8962-a13314b82f53 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96702025-01-15T02:39:39.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2814 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5cb51198-8d9a-4fe0-8962-a13314b82f53 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96712025-01-15T02:39:40.266ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 956c5b12-da7d-4eed-86e1-63be8c90708f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96722025-01-15T02:39:40.266ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 956c5b12-da7d-4eed-86e1-63be8c90708f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96732025-01-15T02:39:40.268ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2100 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 956c5b12-da7d-4eed-86e1-63be8c90708f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
96742025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
96752025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
96762025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
96772025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
96782025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
96792025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
96802025-01-15T02:39:40.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
96812025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
96822025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
96832025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
96842025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96852025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96862025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96872025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
96882025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
96892025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
96902025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
96912025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96922025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
96932025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96942025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
96952025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96962025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
96972025-01-15T02:39:40.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
96982025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96992025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
97002025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97012025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
97022025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97032025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
97042025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
97052025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97062025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97072025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
97082025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97092025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
97102025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
97112025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97122025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97132025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97142025-01-15T02:39:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97152025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
97162025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
97172025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
97182025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
97192025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
97202025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97212025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
97222025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
97232025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
97242025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
97252025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
97262025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97272025-01-15T02:39:40.504ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4067, tv_nsec: 343754721 } })
97282025-01-15T02:39:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
97292025-01-15T02:39:40.505ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
97302025-01-15T02:39:40.505ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.179046ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
97312025-01-15T02:39:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97322025-01-15T02:39:40.505ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4067, tv_nsec: 344605964 } })
97332025-01-15T02:39:40.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97342025-01-15T02:39:40.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
97352025-01-15T02:39:40.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97362025-01-15T02:39:40.507ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97372025-01-15T02:39:40.507ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
97382025-01-15T02:39:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97392025-01-15T02:39:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
97402025-01-15T02:39:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
97412025-01-15T02:39:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97422025-01-15T02:39:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97432025-01-15T02:39:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
97442025-01-15T02:39:40.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97452025-01-15T02:39:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97462025-01-15T02:39:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
97472025-01-15T02:39:40.509ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97482025-01-15T02:39:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97492025-01-15T02:39:40.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97502025-01-15T02:39:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97512025-01-15T02:39:40.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97522025-01-15T02:39:40.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97532025-01-15T02:39:40.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
97542025-01-15T02:39:40.514ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97552025-01-15T02:39:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97562025-01-15T02:39:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
97572025-01-15T02:39:40.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97582025-01-15T02:39:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97592025-01-15T02:39:40.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97602025-01-15T02:39:40.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97612025-01-15T02:39:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97622025-01-15T02:39:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97632025-01-15T02:39:40.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97642025-01-15T02:39:40.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97652025-01-15T02:39:40.551ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97662025-01-15T02:39:40.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
97672025-01-15T02:39:40.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
97682025-01-15T02:39:40.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
97692025-01-15T02:39:40.559ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.994055ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
97702025-01-15T02:39:40.565ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
97712025-01-15T02:39:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
97722025-01-15T02:39:40.565ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97732025-01-15T02:39:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
97742025-01-15T02:39:40.568ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97752025-01-15T02:39:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97762025-01-15T02:39:40.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 65.928908ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97772025-01-15T02:39:40.573ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97782025-01-15T02:39:40.573ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
97792025-01-15T02:39:40.573ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97802025-01-15T02:39:40.574ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4067, tv_nsec: 412864305 } })
97812025-01-15T02:39:40.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
97822025-01-15T02:39:40.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97832025-01-15T02:39:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 69.613438ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
97842025-01-15T02:39:40.574ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
97852025-01-15T02:39:40.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97862025-01-15T02:39:40.575ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.844598ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
97872025-01-15T02:39:40.576ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97882025-01-15T02:39:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97892025-01-15T02:39:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
97902025-01-15T02:39:40.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97912025-01-15T02:39:40.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97922025-01-15T02:39:40.577ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97932025-01-15T02:39:40.578ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97942025-01-15T02:39:40.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
97952025-01-15T02:39:40.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97962025-01-15T02:39:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97972025-01-15T02:39:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97982025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97992025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
98002025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
98012025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98022025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
98032025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
98042025-01-15T02:39:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
98052025-01-15T02:39:40.581ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 76.763502ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
98062025-01-15T02:39:40.582ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98072025-01-15T02:39:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98082025-01-15T02:39:40.585ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
98092025-01-15T02:39:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
98102025-01-15T02:39:40.586ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
98112025-01-15T02:39:40.586ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
98122025-01-15T02:39:40.587ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "59b8582d-5ebe-4e1e-89da-dd8a81478040", "content-length": "26", "date": "Wed, 15 Jan 2025 02:39:39 GMT"} })
98132025-01-15T02:39:40.587ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
98142025-01-15T02:39:40.588ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
98152025-01-15T02:39:40.592ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
98162025-01-15T02:39:40.592ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 88.852548ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
98172025-01-15T02:39:40.593ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
98182025-01-15T02:39:40.593ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 90.054821ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
98192025-01-15T02:39:40.595ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
98202025-01-15T02:39:40.595ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
98212025-01-15T02:39:40.596ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "457dfdf2-0367-4d6d-8da7-8cfebff1b9a1", "content-length": "26", "date": "Wed, 15 Jan 2025 02:39:39 GMT"} })
98222025-01-15T02:39:40.596ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
98232025-01-15T02:39:40.598ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
98242025-01-15T02:39:40.608ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
98252025-01-15T02:39:40.608ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
98262025-01-15T02:39:40.608ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 105.049065ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
98272025-01-15T02:39:40.615ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98282025-01-15T02:39:40.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
98292025-01-15T02:39:40.619ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98302025-01-15T02:39:40.619ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98312025-01-15T02:39:40.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98322025-01-15T02:39:40.621ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
98332025-01-15T02:39:40.621ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
98342025-01-15T02:39:40.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
98352025-01-15T02:39:40.623ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.858914ms iteration = 26 status = 0
98362025-01-15T02:39:40.629ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 125.082855ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
98372025-01-15T02:39:40.644ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
98382025-01-15T02:39:40.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
98392025-01-15T02:39:40.652ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
98402025-01-15T02:39:40.652ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 148.883957ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
98412025-01-15T02:39:40.667ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
98422025-01-15T02:39:40.667ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
98432025-01-15T02:39:40.668ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d56fd7b4-e49d-442a-a004-cc76103060ba", "content-length": "26", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98442025-01-15T02:39:40.668ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
98452025-01-15T02:39:40.670ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
98462025-01-15T02:39:40.701ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 199.424501ms iteration = 26 status = {}
98472025-01-15T02:39:40.711ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 207.876749ms iteration = 29 status = {"physical_disks_added":0}
98482025-01-15T02:39:40.732ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
98492025-01-15T02:39:40.732ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d933b414-c8fe-4eab-b3ca-c7f15ba27937", "content-length": "2", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98502025-01-15T02:39:40.735ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
98512025-01-15T02:39:40.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
98522025-01-15T02:39:40.736ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 234.5748ms iteration = 26 status = null
98532025-01-15T02:39:40.743ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
98542025-01-15T02:39:40.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
98552025-01-15T02:39:40.745ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98562025-01-15T02:39:40.746ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
98572025-01-15T02:39:40.746ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
98582025-01-15T02:39:40.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
98592025-01-15T02:39:40.748ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98602025-01-15T02:39:40.750ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
98612025-01-15T02:39:40.750ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
98622025-01-15T02:39:40.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
98632025-01-15T02:39:40.758ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
98642025-01-15T02:39:40.758ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
98652025-01-15T02:39:40.758ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "355ee1a8-4122-45fc-894b-d341ff7b84bc", "content-length": "372", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98662025-01-15T02:39:40.758ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98672025-01-15T02:39:40.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98682025-01-15T02:39:40.801ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
98692025-01-15T02:39:40.801ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
98702025-01-15T02:39:40.809ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98712025-01-15T02:39:40.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98722025-01-15T02:39:40.812ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
98732025-01-15T02:39:40.812ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98742025-01-15T02:39:40.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98752025-01-15T02:39:40.822ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
98762025-01-15T02:39:40.822ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "229ec34b-54bb-4e04-9ae7-a9ab48014d07", "content-length": "134", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98772025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
98782025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
98792025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
98802025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
98812025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98822025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98832025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98842025-01-15T02:39:40.822ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
98852025-01-15T02:39:40.823ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98862025-01-15T02:39:40.823ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
98872025-01-15T02:39:40.823ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
98882025-01-15T02:39:40.823ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98892025-01-15T02:39:40.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98902025-01-15T02:39:40.825ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
98912025-01-15T02:39:40.825ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "250ca53a-64e8-4259-916f-f2c2bdd9fffa", "content-length": "2", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98922025-01-15T02:39:40.825ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
98932025-01-15T02:39:40.825ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "644be607-c666-46aa-96c1-5d25a9f4732d", "content-length": "52", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98942025-01-15T02:39:40.825ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
98952025-01-15T02:39:40.825ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
98962025-01-15T02:39:40.842ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
98972025-01-15T02:39:40.865ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d6452ee5-fc71-48ac-874d-6e8e82e708fb", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
98982025-01-15T02:39:40.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98992025-01-15T02:39:40.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99002025-01-15T02:39:40.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99012025-01-15T02:39:40.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99022025-01-15T02:39:40.870ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdf5fbe9-2281-4f3e-be64-c3bf7ff19f83", "content-length": "691", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
99032025-01-15T02:39:40.870ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
99042025-01-15T02:39:40.870ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
99052025-01-15T02:39:40.870ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
99062025-01-15T02:39:40.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99072025-01-15T02:39:40.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99082025-01-15T02:39:40.872ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 369.125311ms iteration = 27 status = {}
99092025-01-15T02:39:40.874ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
99102025-01-15T02:39:40.874ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
99112025-01-15T02:39:40.875ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f0a88b7f-3965-43f6-b313-b44a20afabb2", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
99122025-01-15T02:39:40.876ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 372.919423ms iteration = 27 status = {}
99132025-01-15T02:39:40.966ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7c779df0-7c9c-470b-904f-daa162b11b53", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
99142025-01-15T02:39:40.966ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
99152025-01-15T02:39:40.966ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
99162025-01-15T02:39:40.966ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "69851d77-c83c-400d-a52c-d74939f8f575", "content-length": "682", "date": "Wed, 15 Jan 2025 02:39:40 GMT"} })
99172025-01-15T02:39:40.966ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99182025-01-15T02:39:40.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99192025-01-15T02:39:40.970ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99202025-01-15T02:39:40.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99212025-01-15T02:39:40.974ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99222025-01-15T02:39:40.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99232025-01-15T02:39:40.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99242025-01-15T02:39:40.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99252025-01-15T02:39:40.983ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 480.332482ms iteration = 26 status = {}
99262025-01-15T02:39:41.286ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d7742434-4be4-411f-b71c-c4cd84935bc0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99272025-01-15T02:39:41.286ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d7742434-4be4-411f-b71c-c4cd84935bc0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99282025-01-15T02:39:41.288ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2193 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d7742434-4be4-411f-b71c-c4cd84935bc0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99292025-01-15T02:39:42.307ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 335d7762-5608-4951-a017-da21dddc934f roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99302025-01-15T02:39:42.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 335d7762-5608-4951-a017-da21dddc934f resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99312025-01-15T02:39:42.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2249 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 335d7762-5608-4951-a017-da21dddc934f response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99322025-01-15T02:39:43.330ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cefb6693-c5b3-4fbb-af54-efc7fde69736 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99332025-01-15T02:39:43.331ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cefb6693-c5b3-4fbb-af54-efc7fde69736 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99342025-01-15T02:39:43.332ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2046 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = cefb6693-c5b3-4fbb-af54-efc7fde69736 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99352025-01-15T02:39:44.351ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d119c285-ee78-48f2-9bbd-41c03e72c6db roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99362025-01-15T02:39:44.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d119c285-ee78-48f2-9bbd-41c03e72c6db resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99372025-01-15T02:39:44.354ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2539 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d119c285-ee78-48f2-9bbd-41c03e72c6db response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99382025-01-15T02:39:45.374ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ffc1ff5b-d790-40d0-bb16-0d770faaab76 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99392025-01-15T02:39:45.374ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ffc1ff5b-d790-40d0-bb16-0d770faaab76 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99402025-01-15T02:39:45.376ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2452 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ffc1ff5b-d790-40d0-bb16-0d770faaab76 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99412025-01-15T02:39:45.744ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 483 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 073a1f6d-0b83-4b4c-ac3b-2ae9dde47748 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
99422025-01-15T02:39:46.396ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 369fe3b2-8479-4fce-95a4-50488af25db8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99432025-01-15T02:39:46.396ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 369fe3b2-8479-4fce-95a4-50488af25db8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99442025-01-15T02:39:46.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1721 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 369fe3b2-8479-4fce-95a4-50488af25db8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99452025-01-15T02:39:47.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 143a46d5-7b47-4729-8b8f-0fd40c747373 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99462025-01-15T02:39:47.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 143a46d5-7b47-4729-8b8f-0fd40c747373 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99472025-01-15T02:39:47.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2402 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 143a46d5-7b47-4729-8b8f-0fd40c747373 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99482025-01-15T02:39:48.439ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df4be1e2-8558-4c00-80d4-9604dad72177 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99492025-01-15T02:39:48.439ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df4be1e2-8558-4c00-80d4-9604dad72177 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99502025-01-15T02:39:48.441ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2434 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = df4be1e2-8558-4c00-80d4-9604dad72177 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99512025-01-15T02:39:49.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b60f076e-23b9-4b2f-857d-b8d373505cf7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99522025-01-15T02:39:49.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b60f076e-23b9-4b2f-857d-b8d373505cf7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99532025-01-15T02:39:49.463ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2315 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b60f076e-23b9-4b2f-857d-b8d373505cf7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99542025-01-15T02:39:50.482ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7f563511-b1cd-4232-98a5-703d0a46523c roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99552025-01-15T02:39:50.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7f563511-b1cd-4232-98a5-703d0a46523c resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99562025-01-15T02:39:50.484ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2357 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 7f563511-b1cd-4232-98a5-703d0a46523c response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99572025-01-15T02:39:50.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
99582025-01-15T02:39:50.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
99592025-01-15T02:39:50.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99602025-01-15T02:39:50.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99612025-01-15T02:39:50.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99622025-01-15T02:39:50.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99632025-01-15T02:39:50.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99642025-01-15T02:39:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99652025-01-15T02:39:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99662025-01-15T02:39:50.506ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.096581ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
99672025-01-15T02:39:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
99682025-01-15T02:39:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99692025-01-15T02:39:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99702025-01-15T02:39:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99712025-01-15T02:39:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99722025-01-15T02:39:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99732025-01-15T02:39:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
99742025-01-15T02:39:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99752025-01-15T02:39:50.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99762025-01-15T02:39:50.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
99772025-01-15T02:39:50.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.572223ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
99782025-01-15T02:39:51.504ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 94491233-09e3-44bd-a917-0be8f35ff6c3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99792025-01-15T02:39:51.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 94491233-09e3-44bd-a917-0be8f35ff6c3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99802025-01-15T02:39:51.506ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 94491233-09e3-44bd-a917-0be8f35ff6c3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99812025-01-15T02:39:52.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a214d35c-2be4-45a4-acd7-334146ed4cec roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99822025-01-15T02:39:52.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a214d35c-2be4-45a4-acd7-334146ed4cec resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99832025-01-15T02:39:52.529ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2838 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = a214d35c-2be4-45a4-acd7-334146ed4cec response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99842025-01-15T02:39:53.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae414e2e-d276-4043-9a33-665931357157 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99852025-01-15T02:39:53.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae414e2e-d276-4043-9a33-665931357157 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99862025-01-15T02:39:53.550ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2283 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = ae414e2e-d276-4043-9a33-665931357157 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99872025-01-15T02:39:54.569ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 59d6c3cc-ef18-42b7-b958-337becc6a7f7 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99882025-01-15T02:39:54.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 59d6c3cc-ef18-42b7-b958-337becc6a7f7 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99892025-01-15T02:39:54.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2524 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 59d6c3cc-ef18-42b7-b958-337becc6a7f7 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99902025-01-15T02:39:55.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b688f67d-7842-4633-aecb-4a62867b77a8 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99912025-01-15T02:39:55.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b688f67d-7842-4633-aecb-4a62867b77a8 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99922025-01-15T02:39:55.593ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1718 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b688f67d-7842-4633-aecb-4a62867b77a8 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99932025-01-15T02:39:55.744ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 538 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 8b7cc67e-3d44-4a61-b44f-fb6a91b5443a response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
99942025-01-15T02:39:56.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 665d337f-272c-4473-8388-246dcc1ba995 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99952025-01-15T02:39:56.614ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 665d337f-272c-4473-8388-246dcc1ba995 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99962025-01-15T02:39:56.616ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2387 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 665d337f-272c-4473-8388-246dcc1ba995 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99972025-01-15T02:39:57.636ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 861e6b78-5eb6-4a45-ad20-a6d2c4b7f02a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99982025-01-15T02:39:57.636ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 861e6b78-5eb6-4a45-ad20-a6d2c4b7f02a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
99992025-01-15T02:39:57.638ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2884 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 861e6b78-5eb6-4a45-ad20-a6d2c4b7f02a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100002025-01-15T02:39:58.658ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 478db84c-89da-429f-8e10-3e6801cba283 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100012025-01-15T02:39:58.658ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 478db84c-89da-429f-8e10-3e6801cba283 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100022025-01-15T02:39:58.660ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 478db84c-89da-429f-8e10-3e6801cba283 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100032025-01-15T02:39:59.681ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 24bb07b1-c183-44c2-8772-85b38e50728a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100042025-01-15T02:39:59.681ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 24bb07b1-c183-44c2-8772-85b38e50728a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100052025-01-15T02:39:59.683ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1763 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 24bb07b1-c183-44c2-8772-85b38e50728a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100062025-01-15T02:40:00.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
100072025-01-15T02:40:00.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
100082025-01-15T02:40:00.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100092025-01-15T02:40:00.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100102025-01-15T02:40:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100112025-01-15T02:40:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100122025-01-15T02:40:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100132025-01-15T02:40:00.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100142025-01-15T02:40:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100152025-01-15T02:40:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
100162025-01-15T02:40:00.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100172025-01-15T02:40:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100182025-01-15T02:40:00.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.403858ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
100192025-01-15T02:40:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100202025-01-15T02:40:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100212025-01-15T02:40:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100222025-01-15T02:40:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
100232025-01-15T02:40:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100242025-01-15T02:40:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100252025-01-15T02:40:00.522ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
100262025-01-15T02:40:00.522ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.491376ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
100272025-01-15T02:40:00.702ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29764382-b0bd-4231-af1b-34aaa0f0e560 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100282025-01-15T02:40:00.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29764382-b0bd-4231-af1b-34aaa0f0e560 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100292025-01-15T02:40:00.704ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 29764382-b0bd-4231-af1b-34aaa0f0e560 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100302025-01-15T02:40:01.726ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 329e8f2f-5eb4-458d-ae67-9587d76f7156 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100312025-01-15T02:40:01.726ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 329e8f2f-5eb4-458d-ae67-9587d76f7156 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100322025-01-15T02:40:01.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2274 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 329e8f2f-5eb4-458d-ae67-9587d76f7156 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100332025-01-15T02:40:02.749ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4b5bd63b-f2dd-4df0-8fe0-5f9ddf8b56c5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100342025-01-15T02:40:02.750ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4b5bd63b-f2dd-4df0-8fe0-5f9ddf8b56c5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100352025-01-15T02:40:02.752ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2186 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 4b5bd63b-f2dd-4df0-8fe0-5f9ddf8b56c5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100362025-01-15T02:40:03.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 34b5e2ed-447d-436c-8329-68c3203c9b17 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100372025-01-15T02:40:03.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 34b5e2ed-447d-436c-8329-68c3203c9b17 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100382025-01-15T02:40:03.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 34b5e2ed-447d-436c-8329-68c3203c9b17 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100392025-01-15T02:40:04.792ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3fdae1e6-b96c-4f78-95b3-b7cdfd178dfb roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100402025-01-15T02:40:04.792ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3fdae1e6-b96c-4f78-95b3-b7cdfd178dfb resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100412025-01-15T02:40:04.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3fdae1e6-b96c-4f78-95b3-b7cdfd178dfb response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100422025-01-15T02:40:05.744ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 494 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = ff205331-246e-4d9f-86a4-dc2aa3b01467 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
100432025-01-15T02:40:05.813ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c4ebd13-ef88-40e4-81f3-bc67861dea17 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100442025-01-15T02:40:05.814ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c4ebd13-ef88-40e4-81f3-bc67861dea17 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100452025-01-15T02:40:05.816ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2624 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 6c4ebd13-ef88-40e4-81f3-bc67861dea17 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100462025-01-15T02:40:06.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 253d8a4d-f11d-427d-9be1-102e894f5485 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100472025-01-15T02:40:06.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 253d8a4d-f11d-427d-9be1-102e894f5485 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100482025-01-15T02:40:06.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2414 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 253d8a4d-f11d-427d-9be1-102e894f5485 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100492025-01-15T02:40:07.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4af4a7c-5217-41a7-86ec-9158662c6ad5 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100502025-01-15T02:40:07.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4af4a7c-5217-41a7-86ec-9158662c6ad5 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100512025-01-15T02:40:07.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2315 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = d4af4a7c-5217-41a7-86ec-9158662c6ad5 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100522025-01-15T02:40:08.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d016f49-5942-4dfb-8eaa-373a521f132b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100532025-01-15T02:40:08.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d016f49-5942-4dfb-8eaa-373a521f132b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100542025-01-15T02:40:08.883ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2086 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 5d016f49-5942-4dfb-8eaa-373a521f132b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100552025-01-15T02:40:09.902ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0f87b1cd-78a7-4566-a324-931afcf4a260 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100562025-01-15T02:40:09.902ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0f87b1cd-78a7-4566-a324-931afcf4a260 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100572025-01-15T02:40:09.904ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1807 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 0f87b1cd-78a7-4566-a324-931afcf4a260 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
100582025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
100592025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
100602025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
100612025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
100622025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
100632025-01-15T02:40:10.502ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-15 02:30:10.502432711 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
100642025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
100652025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
100662025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
100672025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
100682025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
100692025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
100702025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
100712025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
100722025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100732025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
100742025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
100752025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100762025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
100772025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
100782025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
100792025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
100802025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
100812025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100822025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
100832025-01-15T02:40:10.502ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
100842025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
100852025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
100862025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100872025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
100882025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
100892025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
100902025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
100912025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
100922025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
100932025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
100942025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
100952025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
100962025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
100972025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
100982025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
100992025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101002025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
101012025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101022025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
101032025-01-15T02:40:10.502ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
101042025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
101052025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
101062025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
101072025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
101082025-01-15T02:40:10.502ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 0899079f-636a-4bad-9471-764b8822a779
101092025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.52\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"0899079f-636a-4bad-9471-764b8822a779"}
101102025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
101112025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
101122025-01-15T02:40:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101132025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101142025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
101152025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
101162025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101172025-01-15T02:40:10.503ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e94e881-2bab-4d68-8315-741d3cd1814a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44ac331a-da63-47e3-a636-214c3ca86b22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a4d0b75d-0c67-4224-af19-0510665c8775.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 4097, tv_nsec: 342179388 } })
101182025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
101192025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101202025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101212025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
101222025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101232025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
101242025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
101252025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101262025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101272025-01-15T02:40:10.503ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7468547b-b2f3-4993-afd8-72e9164a4301.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("8b302e6b-b09c-4ef2-8768-1e5fae50d31a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 4097, tv_nsec: 342327332 } })
101282025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
101292025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
101302025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101312025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
101322025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
101332025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
101342025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
101352025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
101362025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
101372025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
101382025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
101392025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
101402025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
101412025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
101422025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
101432025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
101442025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
101452025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.48574ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
101462025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
101472025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.567922ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
101482025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
101492025-01-15T02:40:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
101502025-01-15T02:40:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
101512025-01-15T02:40:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
101522025-01-15T02:40:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
101532025-01-15T02:40:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101542025-01-15T02:40:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101552025-01-15T02:40:10.505ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-15 02:30:10.502432711 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
101562025-01-15T02:40:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.33758ms iteration = 14 status = {"expiration":"2025-01-15T02:30:10.502432711Z","pruned":[]}
101572025-01-15T02:40:10.505ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4097, tv_nsec: 344836350 } })
101582025-01-15T02:40:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101592025-01-15T02:40:10.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
101602025-01-15T02:40:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
101612025-01-15T02:40:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.894816ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
101622025-01-15T02:40:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101632025-01-15T02:40:10.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
101642025-01-15T02:40:10.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101652025-01-15T02:40:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
101662025-01-15T02:40:10.508ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4097, tv_nsec: 347583694 } })
101672025-01-15T02:40:10.508ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
101682025-01-15T02:40:10.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101692025-01-15T02:40:10.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101702025-01-15T02:40:10.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101712025-01-15T02:40:10.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101722025-01-15T02:40:10.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101732025-01-15T02:40:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101742025-01-15T02:40:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
101752025-01-15T02:40:10.513ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101762025-01-15T02:40:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101772025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
101782025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
101792025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
101802025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101812025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
101822025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101832025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
101842025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101852025-01-15T02:40:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101862025-01-15T02:40:10.515ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101872025-01-15T02:40:10.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101882025-01-15T02:40:10.517ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101892025-01-15T02:40:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
101902025-01-15T02:40:10.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101912025-01-15T02:40:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101922025-01-15T02:40:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
101932025-01-15T02:40:10.519ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101942025-01-15T02:40:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
101952025-01-15T02:40:10.519ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
101962025-01-15T02:40:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101972025-01-15T02:40:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
101982025-01-15T02:40:10.521ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
101992025-01-15T02:40:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
102002025-01-15T02:40:10.522ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.593422ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
102012025-01-15T02:40:10.523ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 19.530787ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
102022025-01-15T02:40:10.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102032025-01-15T02:40:10.525ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102042025-01-15T02:40:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 22.582569ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
102052025-01-15T02:40:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102062025-01-15T02:40:10.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102072025-01-15T02:40:10.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ce66c06a-7d54-47a6-860d-59da68080fee.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4097, tv_nsec: 366350740 } })
102082025-01-15T02:40:10.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102092025-01-15T02:40:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102102025-01-15T02:40:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102112025-01-15T02:40:10.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
102122025-01-15T02:40:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
102132025-01-15T02:40:10.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102142025-01-15T02:40:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102152025-01-15T02:40:10.533ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102162025-01-15T02:40:10.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102172025-01-15T02:40:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
102182025-01-15T02:40:10.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102192025-01-15T02:40:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102202025-01-15T02:40:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
102212025-01-15T02:40:10.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102222025-01-15T02:40:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102232025-01-15T02:40:10.545ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102242025-01-15T02:40:10.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
102252025-01-15T02:40:10.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102262025-01-15T02:40:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102272025-01-15T02:40:10.559ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
102282025-01-15T02:40:10.560ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.42762ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}
102292025-01-15T02:40:10.565ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102302025-01-15T02:40:10.565ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
102312025-01-15T02:40:10.565ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102322025-01-15T02:40:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
102332025-01-15T02:40:10.566ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102342025-01-15T02:40:10.567ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102352025-01-15T02:40:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102362025-01-15T02:40:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
102372025-01-15T02:40:10.568ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
102382025-01-15T02:40:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
102392025-01-15T02:40:10.572ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102402025-01-15T02:40:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102412025-01-15T02:40:10.572ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102422025-01-15T02:40:10.572ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
102432025-01-15T02:40:10.572ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
102442025-01-15T02:40:10.572ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 69.179477ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
102452025-01-15T02:40:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102462025-01-15T02:40:10.573ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
102472025-01-15T02:40:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
102482025-01-15T02:40:10.573ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 70.286167ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
102492025-01-15T02:40:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 71.072608ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
102502025-01-15T02:40:10.576ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102512025-01-15T02:40:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102522025-01-15T02:40:10.577ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102532025-01-15T02:40:10.579ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102542025-01-15T02:40:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
102552025-01-15T02:40:10.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102562025-01-15T02:40:10.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102572025-01-15T02:40:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102582025-01-15T02:40:10.582ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102592025-01-15T02:40:10.583ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
102602025-01-15T02:40:10.583ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 80.769769ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
102612025-01-15T02:40:10.583ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
102622025-01-15T02:40:10.583ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
102632025-01-15T02:40:10.583ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
102642025-01-15T02:40:10.583ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102652025-01-15T02:40:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102662025-01-15T02:40:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
102672025-01-15T02:40:10.584ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102682025-01-15T02:40:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102692025-01-15T02:40:10.586ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102702025-01-15T02:40:10.590ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:358
102712025-01-15T02:40:10.590ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 86.393051ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
102722025-01-15T02:40:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
102732025-01-15T02:40:10.593ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102742025-01-15T02:40:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102752025-01-15T02:40:10.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102762025-01-15T02:40:10.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102772025-01-15T02:40:10.609ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102782025-01-15T02:40:10.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
102792025-01-15T02:40:10.613ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102802025-01-15T02:40:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102812025-01-15T02:40:10.616ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102822025-01-15T02:40:10.616ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
102832025-01-15T02:40:10.616ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102842025-01-15T02:40:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
102852025-01-15T02:40:10.620ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
102862025-01-15T02:40:10.620ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
102872025-01-15T02:40:10.620ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 118.147188ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
102882025-01-15T02:40:10.621ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102892025-01-15T02:40:10.623ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 121.073227ms iteration = 30 status = {"physical_disks_added":0}
102902025-01-15T02:40:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ce66c06a-7d54-47a6-860d-59da68080fee, lookup_type: ById(ce66c06a-7d54-47a6-860d-59da68080fee) } result = Ok(())
102912025-01-15T02:40:10.624ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102922025-01-15T02:40:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102932025-01-15T02:40:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 122.365271ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
102942025-01-15T02:40:10.625ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
102952025-01-15T02:40:10.625ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
102962025-01-15T02:40:10.625ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
102972025-01-15T02:40:10.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
102982025-01-15T02:40:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
102992025-01-15T02:40:10.626ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 124.299234ms iteration = 27 status = 0
103002025-01-15T02:40:10.631ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
103012025-01-15T02:40:10.631ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
103022025-01-15T02:40:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
103032025-01-15T02:40:10.633ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
103042025-01-15T02:40:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
103052025-01-15T02:40:10.648ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
103062025-01-15T02:40:10.648ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
103072025-01-15T02:40:10.648ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
103082025-01-15T02:40:10.648ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 146.247037ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
103092025-01-15T02:40:10.650ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103102025-01-15T02:40:10.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103112025-01-15T02:40:10.652ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
103122025-01-15T02:40:10.652ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
103132025-01-15T02:40:10.652ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103142025-01-15T02:40:10.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103152025-01-15T02:40:10.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103162025-01-15T02:40:10.656ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
103172025-01-15T02:40:10.656ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
103182025-01-15T02:40:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
103192025-01-15T02:40:10.656ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103202025-01-15T02:40:10.656ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4c09909-ec44-4df9-b2d0-1a227471db22", "content-length": "26", "date": "Wed, 15 Jan 2025 02:40:09 GMT"} })
103212025-01-15T02:40:10.656ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
103222025-01-15T02:40:10.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103232025-01-15T02:40:10.659ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
103242025-01-15T02:40:10.660ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-15T02:27:07.571946Z, time_modified: 2025-01-15T02:27:07.571946Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, db_silo: Silo { identity: SiloIdentity { id: 25a10c65-c21a-4403-8acf-3f8c60b10e2b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-15T02:27:14.495192Z, time_modified: 2025-01-15T02:27:14.495192Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659" }] }) }
103252025-01-15T02:40:10.660ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 158.128237ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]}},"default_endpoint":{"silo_id":"25a10c65-c21a-4403-8acf-3f8c60b10e2b","tls_certs":["13e6a60a4b89097fc134f4f84c7eba3893adb8485fddb764d0c9655de73a6659"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
103262025-01-15T02:40:10.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103272025-01-15T02:40:10.663ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3d3f963c-3315-48e0-9df4-f2cbfe9d60a6 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
103282025-01-15T02:40:10.663ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
103292025-01-15T02:40:10.663ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-15 02:27:13.226792 UTC dns_group = internal generation = 1
103302025-01-15T02:40:10.663ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 160.639615ms iteration = 14 status = {"generation":1}
103312025-01-15T02:40:10.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
103322025-01-15T02:40:10.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
103332025-01-15T02:40:10.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
103342025-01-15T02:40:10.679ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 176.056801ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
103352025-01-15T02:40:10.695ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
103362025-01-15T02:40:10.695ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 0 version = 2
103372025-01-15T02:40:10.695ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
103382025-01-15T02:40:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
103392025-01-15T02:40:10.697ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103402025-01-15T02:40:10.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103412025-01-15T02:40:10.740ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
103422025-01-15T02:40:10.740ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 237.404706ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
103432025-01-15T02:40:10.759ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
103442025-01-15T02:40:10.759ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
103452025-01-15T02:40:10.759ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8187a3b-94bc-49c9-b769-16bbeae6148b", "content-length": "26", "date": "Wed, 15 Jan 2025 02:40:09 GMT"} })
103462025-01-15T02:40:10.759ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
103472025-01-15T02:40:10.761ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
103482025-01-15T02:40:10.793ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
103492025-01-15T02:40:10.802ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 zone_address = fd00:1122:3344:101::2
103502025-01-15T02:40:10.802ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
103512025-01-15T02:40:10.803ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b893954f-d49a-4b94-8cda-6e702d326e55", "content-length": "26", "date": "Wed, 15 Jan 2025 02:40:09 GMT"} })
103522025-01-15T02:40:10.803ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
103532025-01-15T02:40:10.805ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
103542025-01-15T02:40:10.865ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
103552025-01-15T02:40:10.865ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
103562025-01-15T02:40:10.865ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b9090cd-50aa-43af-8f40-4f0e39b74c95", "content-length": "372", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
103572025-01-15T02:40:10.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
103582025-01-15T02:40:10.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
103592025-01-15T02:40:10.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103602025-01-15T02:40:10.867ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103612025-01-15T02:40:10.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103622025-01-15T02:40:10.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103632025-01-15T02:40:10.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103642025-01-15T02:40:10.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
103652025-01-15T02:40:10.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
103662025-01-15T02:40:10.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
103672025-01-15T02:40:10.871ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103682025-01-15T02:40:10.872ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 69f3b9a8-f865-4142-9c2e-7609f56278a4 dns_zone_name = oxide.test found_so_far = 1 version = 2
103692025-01-15T02:40:10.872ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
103702025-01-15T02:40:10.872ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-15 02:27:14.537753 UTC dns_group = external generation = 2
103712025-01-15T02:40:10.872ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 369.817109ms iteration = 14 status = {"generation":2}
103722025-01-15T02:40:10.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103732025-01-15T02:40:10.873ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103742025-01-15T02:40:10.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103752025-01-15T02:40:10.883ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
103762025-01-15T02:40:10.883ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
103772025-01-15T02:40:10.932ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 430.182038ms iteration = 27 status = {}
103782025-01-15T02:40:10.951ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
103792025-01-15T02:40:10.952ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "f91c93f1-15df-4e1b-b7cb-9db6f547e596", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103802025-01-15T02:40:10.952ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103812025-01-15T02:40:10.953ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2d9dcbd2-d202-46d6-ac4e-72ccea263909", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103822025-01-15T02:40:10.953ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103832025-01-15T02:40:10.955ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "0deae134-428f-4c5b-8c4d-4fa45f79286f", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103842025-01-15T02:40:10.955ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103852025-01-15T02:40:10.958ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
103862025-01-15T02:40:10.958ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 455.845221ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
103872025-01-15T02:40:10.958ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
103882025-01-15T02:40:10.959ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6f0edaf6-5a2d-4b27-946b-c4463bb0624e", "date": "Wed, 15 Jan 2025 02:40:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
103892025-01-15T02:40:10.959ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
103902025-01-15T02:40:10.961ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e52c95bb-e15d-4f42-b746-ef8fd22d65c6", "date": "Wed, 15 Jan 2025 02:40:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
103912025-01-15T02:40:10.961ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
103922025-01-15T02:40:10.961ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
103932025-01-15T02:40:10.962ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "7301d4ce-4f0c-4f6c-8b07-ce56f04f0ba7", "content-length": "2", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
103942025-01-15T02:40:10.962ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
103952025-01-15T02:40:10.962ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 460.209759ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
103962025-01-15T02:40:10.963ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
103972025-01-15T02:40:10.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
103982025-01-15T02:40:10.965ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 463.512787ms iteration = 27 status = null
103992025-01-15T02:40:10.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b865b275-4ab3-459c-95fa-87e9264e615a roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104002025-01-15T02:40:10.973ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b865b275-4ab3-459c-95fa-87e9264e615a resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104012025-01-15T02:40:10.973ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8d178c0a-ee56-48d1-a1fb-3191311eb28d", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104022025-01-15T02:40:10.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104032025-01-15T02:40:10.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104042025-01-15T02:40:10.975ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2068 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = b865b275-4ab3-459c-95fa-87e9264e615a response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104052025-01-15T02:40:10.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104062025-01-15T02:40:10.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104072025-01-15T02:40:10.979ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104082025-01-15T02:40:10.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104092025-01-15T02:40:10.990ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
104102025-01-15T02:40:10.990ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "35bf34eb-42d7-4b05-a962-4ea8869bfc44", "content-length": "691", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104112025-01-15T02:40:10.990ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
104122025-01-15T02:40:10.990ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
104132025-01-15T02:40:10.990ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
104142025-01-15T02:40:10.992ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 490.239699ms iteration = 28 status = {}
104152025-01-15T02:40:10.993ZINFOnexus (BackgroundTasks): VPC route manager sled ce66c06a-7d54-47a6-860d-59da68080fee push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
104162025-01-15T02:40:10.993ZDEBGnexus (BackgroundTasks): client request SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
104172025-01-15T02:40:10.993ZDEBGnexus (BackgroundTasks): client response SledAgent = ce66c06a-7d54-47a6-860d-59da68080fee background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "11de2131-7d3f-4394-b081-76b3760fcdb5", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104182025-01-15T02:40:10.996ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 493.320462ms iteration = 28 status = {}
104192025-01-15T02:40:11.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104202025-01-15T02:40:11.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104212025-01-15T02:40:11.040ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 607b0dc2-3cb5-4be3-acf3-0aebf0a3a211, rack_id: 0c4f466f-f877-43e3-8a10-8b20ac7ed608, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(72b9d496-6be8-4fbf-a216-30734aaa130d) } rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
104222025-01-15T02:40:11.040ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104232025-01-15T02:40:11.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104242025-01-15T02:40:11.048ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
104252025-01-15T02:40:11.048ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd8fbfb3-20ca-4eac-82d9-75d04c3a56b9", "content-length": "134", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104262025-01-15T02:40:11.048ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
104272025-01-15T02:40:11.048ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 72b9d496-6be8-4fbf-a216-30734aaa130d
104282025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
104292025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
104302025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104312025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104322025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104332025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
104342025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104352025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
104362025-01-15T02:40:11.049ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608
104372025-01-15T02:40:11.049ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104382025-01-15T02:40:11.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104392025-01-15T02:40:11.051ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
104402025-01-15T02:40:11.051ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "509e0f32-7c22-4ea9-80f5-d01e0a5d314b", "content-length": "2", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104412025-01-15T02:40:11.052ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
104422025-01-15T02:40:11.052ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d24b287-e33e-4a60-ac44-025064d6f378", "content-length": "52", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104432025-01-15T02:40:11.052ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch_location = Switch0
104442025-01-15T02:40:11.052ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
104452025-01-15T02:40:11.144ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "62fda306-5269-418a-9586-68d947762c4a", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104462025-01-15T02:40:11.144ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 switch = Switch0
104472025-01-15T02:40:11.144ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
104482025-01-15T02:40:11.144ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c4f466f-f877-43e3-8a10-8b20ac7ed608 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0c4043d-0955-46ab-8310-7da3bc2690cc", "content-length": "682", "date": "Wed, 15 Jan 2025 02:40:10 GMT"} })
104492025-01-15T02:40:11.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104502025-01-15T02:40:11.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104512025-01-15T02:40:11.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104522025-01-15T02:40:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104532025-01-15T02:40:11.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104542025-01-15T02:40:11.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104552025-01-15T02:40:11.153ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104562025-01-15T02:40:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104572025-01-15T02:40:11.161ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 658.764057ms iteration = 27 status = {}
104582025-01-15T02:40:11.993ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ef74f6d-05bc-4ef1-915b-c7592bc397e1 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104592025-01-15T02:40:11.993ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ef74f6d-05bc-4ef1-915b-c7592bc397e1 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104602025-01-15T02:40:11.995ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2496 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 8ef74f6d-05bc-4ef1-915b-c7592bc397e1 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104612025-01-15T02:40:13.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f8339bee-9570-4445-ba47-e6a0fcee5cf0 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104622025-01-15T02:40:13.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f8339bee-9570-4445-ba47-e6a0fcee5cf0 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104632025-01-15T02:40:13.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2002 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = f8339bee-9570-4445-ba47-e6a0fcee5cf0 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104642025-01-15T02:40:14.037ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 18da6f27-10d4-4e4d-b619-092c69133547 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104652025-01-15T02:40:14.037ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 18da6f27-10d4-4e4d-b619-092c69133547 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104662025-01-15T02:40:14.039ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2491 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 18da6f27-10d4-4e4d-b619-092c69133547 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104672025-01-15T02:40:15.061ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9038e218-c6d0-4aae-ac12-cf8062cbc466 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104682025-01-15T02:40:15.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9038e218-c6d0-4aae-ac12-cf8062cbc466 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104692025-01-15T02:40:15.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2225 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9038e218-c6d0-4aae-ac12-cf8062cbc466 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104702025-01-15T02:40:15.744ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 455 local_addr = [fd00:1122:3344:101::b]:64722 method = GET remote_addr = [fd00:1122:3344:101::d]:49971 req_id = 3e9a8d23-93b3-4600-a7d0-3aba7dbfcf16 response_code = 200 uri = /b4bce764-527e-4a0d-9efa-46f5760fbb1d
104712025-01-15T02:40:16.084ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89f120e9-8c21-4ad1-bea7-f0af1fed3a05 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104722025-01-15T02:40:16.084ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89f120e9-8c21-4ad1-bea7-f0af1fed3a05 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104732025-01-15T02:40:16.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2234 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 89f120e9-8c21-4ad1-bea7-f0af1fed3a05 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104742025-01-15T02:40:17.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 05be5e31-bfd7-4f8e-9ef0-88d555730d65 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104752025-01-15T02:40:17.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 05be5e31-bfd7-4f8e-9ef0-88d555730d65 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104762025-01-15T02:40:17.107ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2401 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 05be5e31-bfd7-4f8e-9ef0-88d555730d65 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104772025-01-15T02:40:18.128ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9bb99466-0d02-4b3f-8348-2159c2d8c4d4 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104782025-01-15T02:40:18.128ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9bb99466-0d02-4b3f-8348-2159c2d8c4d4 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104792025-01-15T02:40:18.130ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 9bb99466-0d02-4b3f-8348-2159c2d8c4d4 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104802025-01-15T02:40:19.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3d3b74b0-d7c1-4ca5-b1f6-bd106a2470f3 roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104812025-01-15T02:40:19.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3d3b74b0-d7c1-4ca5-b1f6-bd106a2470f3 resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104822025-01-15T02:40:19.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1969 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 3d3b74b0-d7c1-4ca5-b1f6-bd106a2470f3 response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104832025-01-15T02:40:20.174ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2934075a-55d0-4e2a-914b-9dc38c9c6f1b roles = RoleSet { roles: {} } uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104842025-01-15T02:40:20.174ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2934075a-55d0-4e2a-914b-9dc38c9c6f1b resource = Database result = Ok(()) uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104852025-01-15T02:40:20.176ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2300 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:35509 req_id = 2934075a-55d0-4e2a-914b-9dc38c9c6f1b response_code = 200 uri = /probes/ce66c06a-7d54-47a6-860d-59da68080fee
104862025-01-15T02:40:20.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
104872025-01-15T02:40:20.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
104882025-01-15T02:40:20.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104892025-01-15T02:40:20.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
104902025-01-15T02:40:20.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104912025-01-15T02:40:20.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104922025-01-15T02:40:20.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104932025-01-15T02:40:20.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
104942025-01-15T02:40:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104952025-01-15T02:40:20.506ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.264145ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
104962025-01-15T02:40:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
104972025-01-15T02:40:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104982025-01-15T02:40:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104992025-01-15T02:40:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105002025-01-15T02:40:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105012025-01-15T02:40:20.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105022025-01-15T02:40:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 0899079f-636a-4bad-9471-764b8822a779, lookup_type: ById(0899079f-636a-4bad-9471-764b8822a779) } result = Ok(())
105032025-01-15T02:40:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105042025-01-15T02:40:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
105052025-01-15T02:40:20.521ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 0899079f-636a-4bad-9471-764b8822a779 original_time_created = 2025-01-15 02:27:13.094632 UTC target_id = 0899079f-636a-4bad-9471-764b8822a779 time_created = 2025-01-15 02:27:13.094632 UTC
105062025-01-15T02:40:20.521ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.125506ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"0899079f-636a-4bad-9471-764b8822a779","time_created":"2025-01-15T02:27:13.094632Z"}