Max level shown:
12025-01-16T20:00:02.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22025-01-16T20:00:02.604ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
32025-01-16T20:00:02.606ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 306.966386ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
42025-01-16T20:00:02.624ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
52025-01-16T20:00:02.635ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 335.746231ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
62025-01-16T20:00:02.645ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
72025-01-16T20:00:02.645ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82025-01-16T20:00:02.646ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f2034ff-8fbd-4197-8486-31994fb9bab3", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
92025-01-16T20:00:02.646ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102025-01-16T20:00:02.646ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
112025-01-16T20:00:02.659ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
122025-01-16T20:00:02.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 949 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = a7fd33cf-06de-4a22-b78f-5603dad4c686 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
132025-01-16T20:00:02.689ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1112 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f zone_address = fd00:1122:3344:101::2
142025-01-16T20:00:02.689ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
152025-01-16T20:00:02.690ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb57e910-36a7-49b1-933a-ea273896b0c2", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
162025-01-16T20:00:02.690ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1117 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
172025-01-16T20:00:02.690ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
182025-01-16T20:00:02.690ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
192025-01-16T20:00:02.691ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccd079e2-d0e3-402b-a07b-3277aa4a0ff3", "content-length": "581", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
202025-01-16T20:00:02.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
212025-01-16T20:00:02.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222025-01-16T20:00:02.692ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
232025-01-16T20:00:02.693ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242025-01-16T20:00:02.697ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
252025-01-16T20:00:02.697ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
262025-01-16T20:00:02.698ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3f5e00d-b96d-47ca-8da0-130ee59fc0f0", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
272025-01-16T20:00:02.698ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
282025-01-16T20:00:02.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
292025-01-16T20:00:02.699ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
302025-01-16T20:00:02.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
312025-01-16T20:00:02.699ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
322025-01-16T20:00:02.701ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
332025-01-16T20:00:02.701ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 481516d2-3f4d-4dc5-9036-8c14a1fa4fd7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
342025-01-16T20:00:02.701ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
352025-01-16T20:00:02.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
362025-01-16T20:00:02.702ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = f05be43b-5d84-4b95-99be-d6d0a1e3a923 dns_zone_name = oxide.test found_so_far = 1 version = 2
372025-01-16T20:00:02.702ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
382025-01-16T20:00:02.702ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-16 19:49:10.849798 UTC dns_group = external generation = 2
392025-01-16T20:00:02.702ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 402.87982ms iteration = 12 status = {"generation":2}
402025-01-16T20:00:02.703ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
412025-01-16T20:00:02.706ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
422025-01-16T20:00:02.706ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
432025-01-16T20:00:02.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
442025-01-16T20:00:02.709ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {0b88d7a9-aac1-4818-9827-ddf6ae9f4566: {10.151.1.175: {9ce66cd4-bd29-4a83-9575-3dd321711509}, 10.151.1.176: {9ce66cd4-bd29-4a83-9575-3dd321711509}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
452025-01-16T20:00:02.709ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
462025-01-16T20:00:02.709ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7f3dd4bb-cc1f-4002-ac97-df291bd160f4", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
472025-01-16T20:00:02.709ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
482025-01-16T20:00:02.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
492025-01-16T20:00:02.729ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
502025-01-16T20:00:02.729ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 430.138608ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
512025-01-16T20:00:02.816ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
522025-01-16T20:00:02.816ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7e71feaa-da0e-4e1f-90ac-0d25b9e29623", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
532025-01-16T20:00:02.816ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
542025-01-16T20:00:02.818ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a1c681bc-20d1-404e-8c2d-7c0dc8312574", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
552025-01-16T20:00:02.818ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
562025-01-16T20:00:02.819ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e15c65b3-82ef-4f27-96ff-6a3ba7f886fc", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
572025-01-16T20:00:02.819ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
582025-01-16T20:00:02.822ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
592025-01-16T20:00:02.822ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 523.222551ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
602025-01-16T20:00:02.859ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
612025-01-16T20:00:02.859ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "78cbb50c-d1e9-4e42-9546-a05845e4e2c6", "content-length": "1", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
622025-01-16T20:00:02.862ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
632025-01-16T20:00:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
642025-01-16T20:00:02.885ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
652025-01-16T20:00:02.886ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6797ec49-ca6f-4d8f-a113-ebf106d17d9f", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
662025-01-16T20:00:02.886ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
672025-01-16T20:00:02.888ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a3377bf3-d44c-457a-99c8-c94ff1f8cb20", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
682025-01-16T20:00:02.888ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
692025-01-16T20:00:02.891ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
702025-01-16T20:00:02.891ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 592.069688ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
712025-01-16T20:00:02.978ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
722025-01-16T20:00:02.979ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dc189fa-235a-4c36-8632-e7a8504fcc5b", "content-length": "809", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
732025-01-16T20:00:02.979ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(305239) }} sled = buskin
742025-01-16T20:00:02.979ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
752025-01-16T20:00:02.979ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
762025-01-16T20:00:02.979ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 680.583522ms iteration = 23 status = {}
772025-01-16T20:00:02.981ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 681.697872ms iteration = 23 status = {}
782025-01-16T20:00:02.981ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
792025-01-16T20:00:02.982ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 481516d2-3f4d-4dc5-9036-8c14a1fa4fd7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
802025-01-16T20:00:02.982ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
812025-01-16T20:00:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
822025-01-16T20:00:02.982ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-16 19:49:07.686045 UTC dns_group = internal generation = 1
832025-01-16T20:00:02.982ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 682.855462ms iteration = 12 status = {"generation":1}
842025-01-16T20:00:02.983ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 683.828739ms iteration = 23 status = null
852025-01-16T20:00:02.986ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
862025-01-16T20:00:02.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
872025-01-16T20:00:02.991ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
882025-01-16T20:00:02.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
892025-01-16T20:00:02.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
902025-01-16T20:00:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
912025-01-16T20:00:02.998ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
922025-01-16T20:00:02.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
932025-01-16T20:00:03.004ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
942025-01-16T20:00:03.004ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
952025-01-16T20:00:03.004ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
962025-01-16T20:00:03.005ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "16952d75-79cc-4b18-ab98-9220ed4fc246", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
972025-01-16T20:00:03.007ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 708.063954ms iteration = 23 status = {}
982025-01-16T20:00:03.057ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
992025-01-16T20:00:03.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1002025-01-16T20:00:03.061ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6ec20bf0-03fa-4e36-9de7-54dd5e9cfde1, rack_id: 26dc04a7-8537-47d8-965d-0931849f651f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(438946af-d2bd-4eab-8036-57fe8293b9f9) } rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
1012025-01-16T20:00:03.061ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1022025-01-16T20:00:03.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1032025-01-16T20:00:03.080ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1042025-01-16T20:00:03.080ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca140301-536c-4b3c-8b22-58036ead4b25", "content-length": "134", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
1052025-01-16T20:00:03.080ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
1062025-01-16T20:00:03.080ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 438946af-d2bd-4eab-8036-57fe8293b9f9
1072025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
1082025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
1092025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1102025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1112025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1122025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1132025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
1142025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
1152025-01-16T20:00:03.081ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
1162025-01-16T20:00:03.081ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1172025-01-16T20:00:03.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1182025-01-16T20:00:03.084ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1192025-01-16T20:00:03.084ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc534f97-87b2-4e03-87a6-31674b28acc6", "content-length": "2", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
1202025-01-16T20:00:03.084ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1212025-01-16T20:00:03.084ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "460deb26-3c39-4c88-9c6a-9bc8875906bf", "content-length": "52", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
1222025-01-16T20:00:03.084ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
1232025-01-16T20:00:03.084ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1242025-01-16T20:00:03.192ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
1252025-01-16T20:00:03.192ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::a]:12221
1262025-01-16T20:00:03.224ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1272025-01-16T20:00:03.224ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:34554
1282025-01-16T20:00:03.224ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e263084d-fec5-4d84-b9d9-3dc160cbe0b6", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
1292025-01-16T20:00:03.224ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch = Switch0
1302025-01-16T20:00:03.224ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1312025-01-16T20:00:03.224ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:34554 req_id = fb2d5fa7-4dea-4297-8498-ee8634959bff roles = RoleSet { roles: {} } uri = /metrics/producers
1322025-01-16T20:00:03.224ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c55d227-ab53-41da-a7a2-61640ca2914f", "content-length": "682", "date": "Thu, 16 Jan 2025 20:00:03 GMT"} })
1332025-01-16T20:00:03.224ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:34554 req_id = fb2d5fa7-4dea-4297-8498-ee8634959bff resource = Database result = Ok(()) uri = /metrics/producers
1342025-01-16T20:00:03.224ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1352025-01-16T20:00:03.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1362025-01-16T20:00:03.228ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1372025-01-16T20:00:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1382025-01-16T20:00:03.232ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1392025-01-16T20:00:03.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1402025-01-16T20:00:03.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1412025-01-16T20:00:03.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1422025-01-16T20:00:03.241ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 942.091161ms iteration = 23 status = {}
1432025-01-16T20:00:03.262ZINFOnexus: assigned collector to new producer collector_id = d048ac10-211f-4354-a749-75b8d62cfaed file = nexus/src/app/oximeter.rs:76 producer_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
1442025-01-16T20:00:03.262ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37739 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:34554 req_id = fb2d5fa7-4dea-4297-8498-ee8634959bff response_code = 201 uri = /metrics/producers
1452025-01-16T20:00:03.262ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "fb2d5fa7-4dea-4297-8498-ee8634959bff", "content-length": "41", "date": "Thu, 16 Jan 2025 20:00:02 GMT"} })
1462025-01-16T20:00:03.263ZDEBGnexus (producer-registration-task): registered with nexus successfully
1472025-01-16T20:00:03.264ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
1482025-01-16T20:00:03.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 04ff28f7-01e6-4eea-a606-46bc11a4cb8e roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1492025-01-16T20:00:03.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 04ff28f7-01e6-4eea-a606-46bc11a4cb8e resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1502025-01-16T20:00:03.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2426 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 04ff28f7-01e6-4eea-a606-46bc11a4cb8e response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1512025-01-16T20:00:04.646ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 61949ca4-283f-4675-ab5d-0b11cb60690a roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1522025-01-16T20:00:04.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 61949ca4-283f-4675-ab5d-0b11cb60690a resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1532025-01-16T20:00:04.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2464 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 61949ca4-283f-4675-ab5d-0b11cb60690a response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1542025-01-16T20:00:05.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 62a40cbd-a13e-453c-bb15-5967f15e83cb roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1552025-01-16T20:00:05.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 62a40cbd-a13e-453c-bb15-5967f15e83cb resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1562025-01-16T20:00:05.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2394 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 62a40cbd-a13e-453c-bb15-5967f15e83cb response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1572025-01-16T20:00:06.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f897b362-88e5-49a9-8117-661dd9134997 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1582025-01-16T20:00:06.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f897b362-88e5-49a9-8117-661dd9134997 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1592025-01-16T20:00:06.694ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2377 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f897b362-88e5-49a9-8117-661dd9134997 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1602025-01-16T20:00:07.715ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 35f2278e-9d40-4df6-951a-86b71ce5fcc8 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1612025-01-16T20:00:07.715ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 35f2278e-9d40-4df6-951a-86b71ce5fcc8 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1622025-01-16T20:00:07.717ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1768 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 35f2278e-9d40-4df6-951a-86b71ce5fcc8 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1632025-01-16T20:00:08.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = fa9fcb80-f4a9-4bd0-b22c-bb67746af680 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1642025-01-16T20:00:08.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = fa9fcb80-f4a9-4bd0-b22c-bb67746af680 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1652025-01-16T20:00:08.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2674 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = fa9fcb80-f4a9-4bd0-b22c-bb67746af680 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1662025-01-16T20:00:09.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 962d1a5e-adc7-4547-ad32-e57762e1d911 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1672025-01-16T20:00:09.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 962d1a5e-adc7-4547-ad32-e57762e1d911 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1682025-01-16T20:00:09.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1863 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 962d1a5e-adc7-4547-ad32-e57762e1d911 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1692025-01-16T20:00:10.507ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 92a6af21-e052-40a2-a996-c243f3f872d8 remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c2178c86-c0e3-4f77-9faa-76d466054650 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-01-16T20:00:10.506760104Z }
1702025-01-16T20:00:10.507ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c2178c86-c0e3-4f77-9faa-76d466054650 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
1712025-01-16T20:00:10.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c2178c86-c0e3-4f77-9faa-76d466054650 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
1722025-01-16T20:00:10.517ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10708 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c2178c86-c0e3-4f77-9faa-76d466054650 response_code = 204 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
1732025-01-16T20:00:10.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ddb36dc6-6f82-4c3a-97a1-54a69bd2e12d roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1742025-01-16T20:00:10.781ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ddb36dc6-6f82-4c3a-97a1-54a69bd2e12d resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1752025-01-16T20:00:10.783ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2243 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ddb36dc6-6f82-4c3a-97a1-54a69bd2e12d response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1762025-01-16T20:00:11.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 670e3ba4-9ee3-437a-b684-ef1ffdbc5475 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
1772025-01-16T20:00:11.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 670e3ba4-9ee3-437a-b684-ef1ffdbc5475 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
1782025-01-16T20:00:11.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3480 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 670e3ba4-9ee3-437a-b684-ef1ffdbc5475 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
1792025-01-16T20:00:11.077ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 46e5bb50-bb0d-4d75-bb20-762698a17c7c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
1802025-01-16T20:00:11.077ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 46e5bb50-bb0d-4d75-bb20-762698a17c7c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
1812025-01-16T20:00:11.080ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3028 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 46e5bb50-bb0d-4d75-bb20-762698a17c7c response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
1822025-01-16T20:00:11.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 31cf3b80-3cc9-4d98-891d-f4e74dc24bfe roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1832025-01-16T20:00:11.802ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 31cf3b80-3cc9-4d98-891d-f4e74dc24bfe resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1842025-01-16T20:00:11.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2582 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 31cf3b80-3cc9-4d98-891d-f4e74dc24bfe response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
1852025-01-16T20:00:12.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
1862025-01-16T20:00:12.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1872025-01-16T20:00:12.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
1882025-01-16T20:00:12.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1892025-01-16T20:00:12.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1902025-01-16T20:00:12.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1912025-01-16T20:00:12.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1922025-01-16T20:00:12.302ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1932025-01-16T20:00:12.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1942025-01-16T20:00:12.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1952025-01-16T20:00:12.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1962025-01-16T20:00:12.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1972025-01-16T20:00:12.304ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.183941ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1982025-01-16T20:00:12.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1992025-01-16T20:00:12.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2002025-01-16T20:00:12.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2012025-01-16T20:00:12.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
2022025-01-16T20:00:12.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2032025-01-16T20:00:12.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2042025-01-16T20:00:12.321ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
2052025-01-16T20:00:12.321ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.496485ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
2062025-01-16T20:00:12.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 077d3ed9-8525-4f77-8034-0fad9076528c roles = RoleSet { roles: {} } uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
2072025-01-16T20:00:12.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 077d3ed9-8525-4f77-8034-0fad9076528c resource = Database result = Ok(()) uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
2082025-01-16T20:00:12.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2619 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 077d3ed9-8525-4f77-8034-0fad9076528c response_code = 200 uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
2092025-01-16T20:00:12.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = d2927cbf-15f9-491b-b818-8d1896d1d385 roles = RoleSet { roles: {} } uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
2102025-01-16T20:00:12.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = d2927cbf-15f9-491b-b818-8d1896d1d385 resource = Database result = Ok(()) uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
2112025-01-16T20:00:12.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2402 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = d2927cbf-15f9-491b-b818-8d1896d1d385 response_code = 200 uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
2122025-01-16T20:00:12.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 728 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = f9880df8-6728-4cba-9e77-16794371ed69 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2132025-01-16T20:00:12.825ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1eccd36e-dcae-4add-9cbf-c183683ee534 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
2142025-01-16T20:00:12.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1eccd36e-dcae-4add-9cbf-c183683ee534 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
2152025-01-16T20:00:12.827ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1eccd36e-dcae-4add-9cbf-c183683ee534 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
2162025-01-16T20:00:12.962ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 92a6af21-e052-40a2-a996-c243f3f872d8 remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-16T20:00:12.962132388Z }
2172025-01-16T20:00:12.962ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2182025-01-16T20:00:12.962ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2192025-01-16T20:00:12.971ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 92a6af21-e052-40a2-a996-c243f3f872d8 remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8 vmm_needs_update = true
2202025-01-16T20:00:12.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2212025-01-16T20:00:12.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2222025-01-16T20:00:12.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2232025-01-16T20:00:12.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2242025-01-16T20:00:12.976ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2252025-01-16T20:00:12.976ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2262025-01-16T20:00:12.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2272025-01-16T20:00:12.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2282025-01-16T20:00:12.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2292025-01-16T20:00:12.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2302025-01-16T20:00:12.981ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2312025-01-16T20:00:12.982ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2322025-01-16T20:00:12.983ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2332025-01-16T20:00:12.988ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 197fa9bd-2b54-417a-8249-a5354e078485, lookup_type: ById(197fa9bd-2b54-417a-8249-a5354e078485) }, key: 3b7a3701-30a8-4657-a953-08ef3b6f1b29, lookup_type: ById(3b7a3701-30a8-4657-a953-08ef3b6f1b29) }, key: 3de40729-f38e-405c-b30f-63d86ff0cdd0, lookup_type: ById(3de40729-f38e-405c-b30f-63d86ff0cdd0) } result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2342025-01-16T20:00:13.000ZINFOnexus (dropshot_internal): starting update saga for 3de40729-f38e-405c-b30f-63d86ff0cdd0 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1544 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-16T20:00:12.962132388Z }
2352025-01-16T20:00:13.000ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37888 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 response_code = 204 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2362025-01-16T20:00:13.000ZDEBGnexus (dropshot_internal): preparing instance-update saga for 3de40729-f38e-405c-b30f-63d86ff0cdd0... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 86e76410-69a2-4705-bdd7-af3af2fc1309 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2372025-01-16T20:00:13.000ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update
2382025-01-16T20:00:13.000ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2077 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 92a6af21-e052-40a2-a996-c243f3f872d8 remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d2acf62d-7367-4db7-ae40-8c0101714364 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-16T20:00:12.962132388Z }
2392025-01-16T20:00:13.000ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d2acf62d-7367-4db7-ae40-8c0101714364 roles = RoleSet { roles: {} } uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2402025-01-16T20:00:13.001ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d2acf62d-7367-4db7-ae40-8c0101714364 resource = Database result = Ok(()) uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2412025-01-16T20:00:13.005ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"3de40729-f38e-405c-b30f-63d86ff0cdd0","lookup_type":{"ById":"3de40729-f38e-405c-b30f-63d86ff0cdd0"},"parent":{"key":"3b7a3701-30a8-4657-a953-08ef3b6f1b29","lookup_type":{"ById":"3b7a3701-30a8-4657-a953-08ef3b6f1b29"},"parent":{"key":"197fa9bd-2b54-417a-8249-a5354e078485","lookup_type":{"ById":"197fa9bd-2b54-417a-8249-a5354e078485"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2422025-01-16T20:00:13.005ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update
2432025-01-16T20:00:13.009ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update
2442025-01-16T20:00:13.009ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2452025-01-16T20:00:13.009ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2462025-01-16T20:00:13.009ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2472025-01-16T20:00:13.016ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2482025-01-16T20:00:13.016ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2492025-01-16T20:00:13.020ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2502025-01-16T20:00:13.020ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2512025-01-16T20:00:13.028ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2522025-01-16T20:00:13.028ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("33f12c32-19cc-4730-892c-1aac40fc30a8")) node_id = 0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2532025-01-16T20:00:13.032ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2542025-01-16T20:00:13.032ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2552025-01-16T20:00:13.037ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36492 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d2acf62d-7367-4db7-ae40-8c0101714364 response_code = 204 uri = /vmms/92a6af21-e052-40a2-a996-c243f3f872d8
2562025-01-16T20:00:13.049ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update
2572025-01-16T20:00:13.053ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2582025-01-16T20:00:13.054ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2592025-01-16T20:00:13.063ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2602025-01-16T20:00:13.063ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2612025-01-16T20:00:13.065ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2622025-01-16T20:00:13.065ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2632025-01-16T20:00:13.066ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2642025-01-16T20:00:13.066ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2652025-01-16T20:00:13.068ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2662025-01-16T20:00:13.068ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2672025-01-16T20:00:13.069ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2682025-01-16T20:00:13.070ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2692025-01-16T20:00:13.071ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2702025-01-16T20:00:13.074ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 197fa9bd-2b54-417a-8249-a5354e078485, lookup_type: ById(197fa9bd-2b54-417a-8249-a5354e078485) }, key: 3b7a3701-30a8-4657-a953-08ef3b6f1b29, lookup_type: ById(3b7a3701-30a8-4657-a953-08ef3b6f1b29) }, key: 3de40729-f38e-405c-b30f-63d86ff0cdd0, lookup_type: ById(3de40729-f38e-405c-b30f-63d86ff0cdd0) } result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2712025-01-16T20:00:13.079ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance updater_id = 33f12c32-19cc-4730-892c-1aac40fc30a8
2722025-01-16T20:00:13.079ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2732025-01-16T20:00:13.079ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2742025-01-16T20:00:13.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 886b0b56-8b8b-47de-927b-2af31c730593 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
2752025-01-16T20:00:13.083ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 886b0b56-8b8b-47de-927b-2af31c730593 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
2762025-01-16T20:00:13.084ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2083 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 886b0b56-8b8b-47de-927b-2af31c730593 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
2772025-01-16T20:00:13.093ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance updater_id = 33f12c32-19cc-4730-892c-1aac40fc30a8
2782025-01-16T20:00:13.093ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2792025-01-16T20:00:13.093ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance
2802025-01-16T20:00:13.097ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 locked_gen = Generation(Generation(2)) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = LockInstance updater_id = 33f12c32-19cc-4730-892c-1aac40fc30a8
2812025-01-16T20:00:13.098ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2822025-01-16T20:00:13.098ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("33f12c32-19cc-4730-892c-1aac40fc30a8")}) node_id = 1 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2832025-01-16T20:00:13.102ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2842025-01-16T20:00:13.102ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
2852025-01-16T20:00:13.107ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2862025-01-16T20:00:13.107ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2872025-01-16T20:00:13.108ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2882025-01-16T20:00:13.108ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2892025-01-16T20:00:13.110ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2902025-01-16T20:00:13.110ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2912025-01-16T20:00:13.111ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2922025-01-16T20:00:13.114ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 197fa9bd-2b54-417a-8249-a5354e078485, lookup_type: ById(197fa9bd-2b54-417a-8249-a5354e078485) }, key: 3b7a3701-30a8-4657-a953-08ef3b6f1b29, lookup_type: ById(3b7a3701-30a8-4657-a953-08ef3b6f1b29) }, key: 3de40729-f38e-405c-b30f-63d86ff0cdd0, lookup_type: ById(3de40729-f38e-405c-b30f-63d86ff0cdd0) } result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2932025-01-16T20:00:13.114ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2942025-01-16T20:00:13.114ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2952025-01-16T20:00:13.125ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 92a6af21-e052-40a2-a996-c243f3f872d8, time_created: 2025-01-16T19:59:52.315441Z, time_deleted: None, instance_id: 3de40729-f38e-405c-b30f-63d86ff0cdd0, sled_id: 87207105-a825-4eaa-ac6d-6aaa586b0fc5, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-01-16T20:00:12.962132Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-01-16T19:59:49.868141Z, gen: Generation(Generation(3)), propolis_id: Some(92a6af21-e052-40a2-a996-c243f3f872d8), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(92a6af21-e052-40a2-a996-c243f3f872d8 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-01-16T20:00:13.125742069Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
2962025-01-16T20:00:13.135ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
2972025-01-16T20:00:13.135ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"3de40729-f38e-405c-b30f-63d86ff0cdd0","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"92a6af21-e052-40a2-a996-c243f3f872d8"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"3de40729-f38e-405c-b30f-63d86ff0cdd0","lookup_type":{"ById":"3de40729-f38e-405c-b30f-63d86ff0cdd0"},"parent":{"key":"3b7a3701-30a8-4657-a953-08ef3b6f1b29","lookup_type":{"ById":"3b7a3701-30a8-4657-a953-08ef3b6f1b29"},"parent":{"key":"197fa9bd-2b54-417a-8249-a5354e078485","lookup_type":{"ById":"197fa9bd-2b54-417a-8249-a5354e078485"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"33f12c32-19cc-4730-892c-1aac40fc30a8"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"3b7a3701-30a8-4657-a953-08ef3b6f1b29","ram_diff":1073741824},"destroy_active_vmm":"92a6af21-e052-40a2-a996-c243f3f872d8","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-01-16T20:00:13.125742069Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
2982025-01-16T20:00:13.135ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
2992025-01-16T20:00:13.167ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3002025-01-16T20:00:13.167ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3012025-01-16T20:00:13.167ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3022025-01-16T20:00:13.167ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3032025-01-16T20:00:13.171ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3042025-01-16T20:00:13.171ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3052025-01-16T20:00:13.175ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3062025-01-16T20:00:13.175ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3072025-01-16T20:00:13.178ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3082025-01-16T20:00:13.178ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("d0028a96-f308-48c4-8700-3b8531fb980d")) node_id = 0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3092025-01-16T20:00:13.181ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3102025-01-16T20:00:13.181ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3112025-01-16T20:00:13.192ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 parent_lock = UpdaterLock { updater_id: 33f12c32-19cc-4730-892c-1aac40fc30a8, locked_gen: Generation(Generation(2)) } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3122025-01-16T20:00:13.192ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = BecomeUpdater
3132025-01-16T20:00:13.192ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = BecomeUpdater
3142025-01-16T20:00:13.208ZDEBGnexus (SagaExecutor): inherited lock from 33f12c32-19cc-4730-892c-1aac40fc30a8 to d0028a96-f308-48c4-8700-3b8531fb980d actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 33f12c32-19cc-4730-892c-1aac40fc30a8 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = BecomeUpdater updater_id = d0028a96-f308-48c4-8700-3b8531fb980d
3152025-01-16T20:00:13.208ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3162025-01-16T20:00:13.208ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3172025-01-16T20:00:13.208ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("d0028a96-f308-48c4-8700-3b8531fb980d")}) node_id = 1 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3182025-01-16T20:00:13.212ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3192025-01-16T20:00:13.212ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3202025-01-16T20:00:13.215ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3212025-01-16T20:00:13.215ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3222025-01-16T20:00:13.219ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3232025-01-16T20:00:13.219ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3242025-01-16T20:00:13.222ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3252025-01-16T20:00:13.222ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0
3262025-01-16T20:00:13.222ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UpdateNetworkConfig
3272025-01-16T20:00:13.223ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UpdateNetworkConfig
3282025-01-16T20:00:13.227ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UpdateNetworkConfig
3292025-01-16T20:00:13.228ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UpdateNetworkConfig
3302025-01-16T20:00:13.233ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
3312025-01-16T20:00:13.233ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UpdateNetworkConfig
3322025-01-16T20:00:13.234ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UpdateNetworkConfig
3332025-01-16T20:00:13.236ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
3342025-01-16T20:00:13.236ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3352025-01-16T20:00:13.236ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3362025-01-16T20:00:13.237ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
3372025-01-16T20:00:13.238ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
3382025-01-16T20:00:13.238ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3392025-01-16T20:00:13.238ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3402025-01-16T20:00:13.244ZDEBGnexus: notifying dendrite of updates instance_id = Some(3de40729-f38e-405c-b30f-63d86ff0cdd0 (instance)) switch = switch0
3412025-01-16T20:00:13.244ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1571, tv_nsec: 478114390 } })
3422025-01-16T20:00:13.245ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1104
3432025-01-16T20:00:13.275ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
3442025-01-16T20:00:13.275ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3452025-01-16T20:00:13.275ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "63fe3337-04ca-4555-a8cd-d5ea6bc6b0c4", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:13 GMT"} })
3462025-01-16T20:00:13.275ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3472025-01-16T20:00:13.276ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3482025-01-16T20:00:13.306ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
3492025-01-16T20:00:13.306ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "e52a5687-e73e-4bc6-aa10-e18a4926c3b3", "content-length": "4", "date": "Thu, 16 Jan 2025 20:00:12 GMT"} })
3502025-01-16T20:00:13.307ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3512025-01-16T20:00:13.307ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3522025-01-16T20:00:13.310ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3532025-01-16T20:00:13.310ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3542025-01-16T20:00:13.316ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
3552025-01-16T20:00:13.316ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
3562025-01-16T20:00:13.329ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-01-16T20:00:13.319283Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 197fa9bd-2b54-417a-8249-a5354e078485, time_modified: Some(2025-01-16T20:00:13.319283Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 3b7a3701-30a8-4657-a953-08ef3b6f1b29, time_modified: Some(2025-01-16T20:00:13.319283Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3572025-01-16T20:00:13.329ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3582025-01-16T20:00:13.329ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3592025-01-16T20:00:13.333ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3602025-01-16T20:00:13.333ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3612025-01-16T20:00:13.336ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3622025-01-16T20:00:13.336ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UnassignOximeterProducer
3632025-01-16T20:00:13.337ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UnassignOximeterProducer
3642025-01-16T20:00:13.344ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = d048ac10-211f-4354-a749-75b8d62cfaed producer_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3652025-01-16T20:00:13.344ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UnassignOximeterProducer
3662025-01-16T20:00:13.344ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = UnassignOximeterProducer
3672025-01-16T20:00:13.384ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3682025-01-16T20:00:13.389ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/3de40729-f38e-405c-b30f-63d86ff0cdd0
3692025-01-16T20:00:13.411ZDEBGnexus (SagaExecutor): client response oximeter-collector = 3de40729-f38e-405c-b30f-63d86ff0cdd0 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/3de40729-f38e-405c-b30f-63d86ff0cdd0", status: 204, headers: {"x-request-id": "3f065302-adae-41a0-8c4a-2bf2fce5e79c", "date": "Thu, 16 Jan 2025 20:00:13 GMT"} }) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3702025-01-16T20:00:13.411ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = d048ac10-211f-4354-a749-75b8d62cfaed producer_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3712025-01-16T20:00:13.412ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3722025-01-16T20:00:13.412ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3732025-01-16T20:00:13.416ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3742025-01-16T20:00:13.416ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3752025-01-16T20:00:13.419ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 lock = UpdaterLock { updater_id: d0028a96-f308-48c4-8700-3b8531fb980d, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-01-16T20:00:13.125742069Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3762025-01-16T20:00:13.419ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = CommitInstanceUpdates
3772025-01-16T20:00:13.419ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = CommitInstanceUpdates
3782025-01-16T20:00:13.489ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 new_runtime = InstanceRuntimeState { time_updated: 2025-01-16T20:00:13.125742069Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
3792025-01-16T20:00:13.489ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3802025-01-16T20:00:13.489ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3812025-01-16T20:00:13.489ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
3822025-01-16T20:00:13.489ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
3832025-01-16T20:00:13.489ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3842025-01-16T20:00:13.489ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3852025-01-16T20:00:13.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3862025-01-16T20:00:13.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3872025-01-16T20:00:13.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3882025-01-16T20:00:13.491ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3892025-01-16T20:00:13.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3902025-01-16T20:00:13.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3912025-01-16T20:00:13.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3922025-01-16T20:00:13.493ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3932025-01-16T20:00:13.493ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
3942025-01-16T20:00:13.494ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3952025-01-16T20:00:13.496ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3962025-01-16T20:00:13.496ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3972025-01-16T20:00:13.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3982025-01-16T20:00:13.497ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
3992025-01-16T20:00:13.497ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("3de40729-f38e-405c-b30f-63d86ff0cdd0"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("92a6af21-e052-40a2-a996-c243f3f872d8")}) node_id = 7 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4002025-01-16T20:00:13.500ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4012025-01-16T20:00:13.500ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4022025-01-16T20:00:13.505ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4032025-01-16T20:00:13.505ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4042025-01-16T20:00:13.509ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4052025-01-16T20:00:13.509ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4062025-01-16T20:00:13.513ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 propolis_id = 92a6af21-e052-40a2-a996-c243f3f872d8 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
4072025-01-16T20:00:13.513ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ReleaseSledResources
4082025-01-16T20:00:13.513ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ReleaseSledResources
4092025-01-16T20:00:13.530ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4102025-01-16T20:00:13.530ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4112025-01-16T20:00:13.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4122025-01-16T20:00:13.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4132025-01-16T20:00:13.530ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7ef5ca5-14eb-4849-ad3a-d70ac093a3c3", "content-length": "372", "date": "Thu, 16 Jan 2025 20:00:13 GMT"} })
4142025-01-16T20:00:13.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4152025-01-16T20:00:13.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4162025-01-16T20:00:13.535ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4172025-01-16T20:00:13.535ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4182025-01-16T20:00:13.538ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4192025-01-16T20:00:13.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4202025-01-16T20:00:13.539ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4212025-01-16T20:00:13.539ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4222025-01-16T20:00:13.540ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4232025-01-16T20:00:13.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4242025-01-16T20:00:13.542ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4252025-01-16T20:00:13.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4262025-01-16T20:00:13.543ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 propolis_id = 92a6af21-e052-40a2-a996-c243f3f872d8 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
4272025-01-16T20:00:13.543ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = MarkVmmDeleted
4282025-01-16T20:00:13.543ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4292025-01-16T20:00:13.543ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = MarkVmmDeleted
4302025-01-16T20:00:13.544ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4312025-01-16T20:00:13.544ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4322025-01-16T20:00:13.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4332025-01-16T20:00:13.552ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4342025-01-16T20:00:13.552ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4352025-01-16T20:00:13.555ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4362025-01-16T20:00:13.555ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4372025-01-16T20:00:13.559ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4382025-01-16T20:00:13.559ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4392025-01-16T20:00:13.562ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4402025-01-16T20:00:13.562ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4412025-01-16T20:00:13.565ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4422025-01-16T20:00:13.565ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4432025-01-16T20:00:13.575ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {0b88d7a9-aac1-4818-9827-ddf6ae9f4566: {10.151.1.176: {9ce66cd4-bd29-4a83-9575-3dd321711509}, 10.151.1.175: {9ce66cd4-bd29-4a83-9575-3dd321711509}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4442025-01-16T20:00:13.575ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4452025-01-16T20:00:13.575ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ac78845f-7b6f-42fa-ad5f-9c930eeb74bd", "date": "Thu, 16 Jan 2025 20:00:13 GMT"} })
4462025-01-16T20:00:13.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4472025-01-16T20:00:13.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4482025-01-16T20:00:13.577ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4492025-01-16T20:00:13.577ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1648b0bc-1b5d-4841-8c37-e07837328319", "content-length": "691", "date": "Thu, 16 Jan 2025 20:00:13 GMT"} })
4502025-01-16T20:00:13.577ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
4512025-01-16T20:00:13.577ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4522025-01-16T20:00:13.577ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4532025-01-16T20:00:13.579ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4542025-01-16T20:00:13.579ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4552025-01-16T20:00:13.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4562025-01-16T20:00:13.579ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4572025-01-16T20:00:13.580ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 90.752262ms iteration = 24 status = {}
4582025-01-16T20:00:13.580ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4592025-01-16T20:00:13.581ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4602025-01-16T20:00:13.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4612025-01-16T20:00:13.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4622025-01-16T20:00:13.582ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4632025-01-16T20:00:13.585ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4642025-01-16T20:00:13.585ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 197fa9bd-2b54-417a-8249-a5354e078485, lookup_type: ById(197fa9bd-2b54-417a-8249-a5354e078485) }, key: 3b7a3701-30a8-4657-a953-08ef3b6f1b29, lookup_type: ById(3b7a3701-30a8-4657-a953-08ef3b6f1b29) }, key: 3de40729-f38e-405c-b30f-63d86ff0cdd0, lookup_type: ById(3de40729-f38e-405c-b30f-63d86ff0cdd0) } result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4652025-01-16T20:00:13.585ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4662025-01-16T20:00:13.585ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4672025-01-16T20:00:13.585ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_node = ChainSuccessorSaga
4682025-01-16T20:00:13.585ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fdb5aff8-1a70-43ae-b49a-2fbbad2631e4", "date": "Thu, 16 Jan 2025 20:00:13 GMT"} })
4692025-01-16T20:00:13.587ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 97.846737ms iteration = 24 status = {}
4702025-01-16T20:00:13.591ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 karmic_state = WillReincarnate needs_reincarnation = false saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update
4712025-01-16T20:00:13.591ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4722025-01-16T20:00:13.591ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4732025-01-16T20:00:13.594ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4742025-01-16T20:00:13.594ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4752025-01-16T20:00:13.598ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4762025-01-16T20:00:13.598ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4772025-01-16T20:00:13.601ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4782025-01-16T20:00:13.601ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936
4792025-01-16T20:00:13.607ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4802025-01-16T20:00:13.610ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 saga_name = instance-update saga_result = SagaResult { saga_id: 9f7a0cf4-769f-4f83-b37e-fcc491eb3936, saga_log: SagaLog { saga_id: 9f7a0cf4-769f-4f83-b37e-fcc491eb3936, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("d0028a96-f308-48c4-8700-3b8531fb980d")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("d0028a96-f308-48c4-8700-3b8531fb980d")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("3de40729-f38e-405c-b30f-63d86ff0cdd0"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("92a6af21-e052-40a2-a996-c243f3f872d8")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("3de40729-f38e-405c-b30f-63d86ff0cdd0"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("92a6af21-e052-40a2-a996-c243f3f872d8")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("d0028a96-f308-48c4-8700-3b8531fb980d"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("d0028a96-f308-48c4-8700-3b8531fb980d")}} }) }
4812025-01-16T20:00:13.610ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 9f7a0cf4-769f-4f83-b37e-fcc491eb3936 instance_id = 3de40729-f38e-405c-b30f-63d86ff0cdd0 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update
4822025-01-16T20:00:13.610ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4832025-01-16T20:00:13.610ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
4842025-01-16T20:00:13.613ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4852025-01-16T20:00:13.613ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
4862025-01-16T20:00:13.616ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4872025-01-16T20:00:13.616ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
4882025-01-16T20:00:13.620ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4892025-01-16T20:00:13.620ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = b068c374-b9ee-4037-8924-b732fa6397a8
4902025-01-16T20:00:13.651ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update sec_id = 4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
4912025-01-16T20:00:13.651ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = b068c374-b9ee-4037-8924-b732fa6397a8 saga_name = start-instance-update saga_result = SagaResult { saga_id: b068c374-b9ee-4037-8924-b732fa6397a8, saga_log: SagaLog { saga_id: b068c374-b9ee-4037-8924-b732fa6397a8, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("33f12c32-19cc-4730-892c-1aac40fc30a8")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("33f12c32-19cc-4730-892c-1aac40fc30a8")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("33f12c32-19cc-4730-892c-1aac40fc30a8"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("33f12c32-19cc-4730-892c-1aac40fc30a8")}} }) }
4922025-01-16T20:00:13.848ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 45b3a93c-5118-4af6-954c-065b92361c7b roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
4932025-01-16T20:00:13.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 45b3a93c-5118-4af6-954c-065b92361c7b resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
4942025-01-16T20:00:13.850ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2060 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 45b3a93c-5118-4af6-954c-065b92361c7b response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
4952025-01-16T20:00:14.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = ad2cff0e-34de-4d17-920a-aa87d2c29d71 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
4962025-01-16T20:00:14.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = ad2cff0e-34de-4d17-920a-aa87d2c29d71 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
4972025-01-16T20:00:14.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3295 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = ad2cff0e-34de-4d17-920a-aa87d2c29d71 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
4982025-01-16T20:00:14.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4fc2ea49-f11c-4985-a3a3-55cb09dff3ad roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
4992025-01-16T20:00:14.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4fc2ea49-f11c-4985-a3a3-55cb09dff3ad resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5002025-01-16T20:00:14.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2172 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4fc2ea49-f11c-4985-a3a3-55cb09dff3ad response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5012025-01-16T20:00:15.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = c63f31fc-04a5-45fd-880b-9cdf96bf5dec roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
5022025-01-16T20:00:15.091ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = c63f31fc-04a5-45fd-880b-9cdf96bf5dec resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
5032025-01-16T20:00:15.093ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2503 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = c63f31fc-04a5-45fd-880b-9cdf96bf5dec response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
5042025-01-16T20:00:15.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 422541a0-76f1-489b-8c64-073b954258ec roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5052025-01-16T20:00:15.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 422541a0-76f1-489b-8c64-073b954258ec resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5062025-01-16T20:00:15.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1904 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 422541a0-76f1-489b-8c64-073b954258ec response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5072025-01-16T20:00:16.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2c531e58-59df-4899-a05e-84c63b9fc855 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5082025-01-16T20:00:16.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2c531e58-59df-4899-a05e-84c63b9fc855 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5092025-01-16T20:00:16.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2183 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2c531e58-59df-4899-a05e-84c63b9fc855 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5102025-01-16T20:00:17.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 78be4b82-adf6-46af-a507-4b076e584871 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5112025-01-16T20:00:17.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 78be4b82-adf6-46af-a507-4b076e584871 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5122025-01-16T20:00:17.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2944 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 78be4b82-adf6-46af-a507-4b076e584871 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5132025-01-16T20:00:18.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = db25d10a-9a22-452b-8359-f83a1de3c552 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5142025-01-16T20:00:18.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = db25d10a-9a22-452b-8359-f83a1de3c552 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5152025-01-16T20:00:18.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2066 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = db25d10a-9a22-452b-8359-f83a1de3c552 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5162025-01-16T20:00:19.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e93a5d1f-56b8-4533-8837-70a0c39cc75b roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5172025-01-16T20:00:19.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e93a5d1f-56b8-4533-8837-70a0c39cc75b resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5182025-01-16T20:00:19.982ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2581 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e93a5d1f-56b8-4533-8837-70a0c39cc75b response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5192025-01-16T20:00:21.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = bf31f83c-11ec-4072-9317-fea77f51a5de roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5202025-01-16T20:00:21.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = bf31f83c-11ec-4072-9317-fea77f51a5de resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5212025-01-16T20:00:21.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2778 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = bf31f83c-11ec-4072-9317-fea77f51a5de response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5222025-01-16T20:00:22.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 443cd5ca-5181-455a-82c8-c647b607fcc3 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5232025-01-16T20:00:22.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 443cd5ca-5181-455a-82c8-c647b607fcc3 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5242025-01-16T20:00:22.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2337 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 443cd5ca-5181-455a-82c8-c647b607fcc3 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5252025-01-16T20:00:22.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
5262025-01-16T20:00:22.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
5272025-01-16T20:00:22.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5282025-01-16T20:00:22.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5292025-01-16T20:00:22.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5302025-01-16T20:00:22.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5312025-01-16T20:00:22.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5322025-01-16T20:00:22.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5332025-01-16T20:00:22.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5342025-01-16T20:00:22.303ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.080352ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5352025-01-16T20:00:22.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5362025-01-16T20:00:22.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5372025-01-16T20:00:22.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5382025-01-16T20:00:22.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5392025-01-16T20:00:22.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5402025-01-16T20:00:22.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5412025-01-16T20:00:22.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
5422025-01-16T20:00:22.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5432025-01-16T20:00:22.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5442025-01-16T20:00:22.318ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
5452025-01-16T20:00:22.318ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.667715ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
5462025-01-16T20:00:22.679ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 939 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 4c964f75-de90-4178-b5a6-18bfced53df0 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
5472025-01-16T20:00:23.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d0448567-f83f-439c-9c71-271b70c26148 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5482025-01-16T20:00:23.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d0448567-f83f-439c-9c71-271b70c26148 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5492025-01-16T20:00:23.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d0448567-f83f-439c-9c71-271b70c26148 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5502025-01-16T20:00:24.066ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f03fecd0-6d95-4754-8452-c3e2f46f31f3 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5512025-01-16T20:00:24.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f03fecd0-6d95-4754-8452-c3e2f46f31f3 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5522025-01-16T20:00:24.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2381 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f03fecd0-6d95-4754-8452-c3e2f46f31f3 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5532025-01-16T20:00:25.088ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8d1798f7-ce4b-4b6b-81ae-155fef036d0e roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5542025-01-16T20:00:25.088ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8d1798f7-ce4b-4b6b-81ae-155fef036d0e resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5552025-01-16T20:00:25.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2327 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8d1798f7-ce4b-4b6b-81ae-155fef036d0e response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5562025-01-16T20:00:26.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 47bcb8a4-2d8e-4b8d-a33b-68525fb72601 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5572025-01-16T20:00:26.108ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 47bcb8a4-2d8e-4b8d-a33b-68525fb72601 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5582025-01-16T20:00:26.110ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1686 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 47bcb8a4-2d8e-4b8d-a33b-68525fb72601 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5592025-01-16T20:00:27.129ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a0220790-651f-41b7-89cc-56e6da7e0729 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5602025-01-16T20:00:27.129ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a0220790-651f-41b7-89cc-56e6da7e0729 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5612025-01-16T20:00:27.132ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2678 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a0220790-651f-41b7-89cc-56e6da7e0729 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5622025-01-16T20:00:28.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = cdfbbade-0ca5-4669-b3c0-98d25911b291 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5632025-01-16T20:00:28.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = cdfbbade-0ca5-4669-b3c0-98d25911b291 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5642025-01-16T20:00:28.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2632 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = cdfbbade-0ca5-4669-b3c0-98d25911b291 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5652025-01-16T20:00:29.171ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1508ceb8-a5dd-49ff-9325-31a2ebc42662 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5662025-01-16T20:00:29.172ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1508ceb8-a5dd-49ff-9325-31a2ebc42662 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5672025-01-16T20:00:29.174ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2382 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1508ceb8-a5dd-49ff-9325-31a2ebc42662 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5682025-01-16T20:00:30.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f4f973a9-6e2d-4592-bf07-9f41701c1279 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5692025-01-16T20:00:30.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f4f973a9-6e2d-4592-bf07-9f41701c1279 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5702025-01-16T20:00:30.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2176 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f4f973a9-6e2d-4592-bf07-9f41701c1279 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5712025-01-16T20:00:31.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 482471c7-c43e-45f0-b5eb-ae8175049a30 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5722025-01-16T20:00:31.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 482471c7-c43e-45f0-b5eb-ae8175049a30 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5732025-01-16T20:00:31.216ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2295 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 482471c7-c43e-45f0-b5eb-ae8175049a30 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5742025-01-16T20:00:32.235ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7d4d42f8-44ae-4045-9319-4093d1fac2c9 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5752025-01-16T20:00:32.235ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7d4d42f8-44ae-4045-9319-4093d1fac2c9 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5762025-01-16T20:00:32.237ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2726 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7d4d42f8-44ae-4045-9319-4093d1fac2c9 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
5772025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
5782025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
5792025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5802025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
5812025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5822025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5832025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
5842025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5852025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
5862025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5872025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
5882025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
5892025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5902025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5912025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
5922025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5932025-01-16T20:00:32.298ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5942025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
5952025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5962025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
5972025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
5982025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5992025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
6002025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6012025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
6022025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
6032025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6042025-01-16T20:00:32.299ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6052025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6062025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6072025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6082025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6092025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6102025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6112025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6122025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6132025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6142025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6152025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6162025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6172025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6182025-01-16T20:00:32.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6192025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
6202025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
6212025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
6222025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6232025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6242025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6252025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
6262025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6272025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6282025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6292025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6302025-01-16T20:00:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6312025-01-16T20:00:32.302ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6322025-01-16T20:00:32.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6332025-01-16T20:00:32.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6342025-01-16T20:00:32.303ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.696113ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6352025-01-16T20:00:32.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6362025-01-16T20:00:32.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6372025-01-16T20:00:32.304ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6382025-01-16T20:00:32.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6392025-01-16T20:00:32.304ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 537894252 } })
6402025-01-16T20:00:32.305ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6412025-01-16T20:00:32.305ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1104
6422025-01-16T20:00:32.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6432025-01-16T20:00:32.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6442025-01-16T20:00:32.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6452025-01-16T20:00:32.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6462025-01-16T20:00:32.306ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6472025-01-16T20:00:32.307ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6482025-01-16T20:00:32.307ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6492025-01-16T20:00:32.307ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6502025-01-16T20:00:32.307ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6512025-01-16T20:00:32.307ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6522025-01-16T20:00:32.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6532025-01-16T20:00:32.308ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 541155119 } })
6542025-01-16T20:00:32.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6552025-01-16T20:00:32.308ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6562025-01-16T20:00:32.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6572025-01-16T20:00:32.308ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1104
6582025-01-16T20:00:32.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6592025-01-16T20:00:32.309ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6602025-01-16T20:00:32.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6612025-01-16T20:00:32.310ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6622025-01-16T20:00:32.310ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6632025-01-16T20:00:32.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6642025-01-16T20:00:32.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6652025-01-16T20:00:32.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6662025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6672025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6682025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6692025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6702025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6712025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6722025-01-16T20:00:32.311ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.764356ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
6732025-01-16T20:00:32.312ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6742025-01-16T20:00:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6752025-01-16T20:00:32.312ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6762025-01-16T20:00:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6772025-01-16T20:00:32.312ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.430858ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
6782025-01-16T20:00:32.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 15.298848ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6792025-01-16T20:00:32.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6802025-01-16T20:00:32.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6812025-01-16T20:00:32.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6822025-01-16T20:00:32.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6832025-01-16T20:00:32.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6842025-01-16T20:00:32.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6852025-01-16T20:00:32.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6862025-01-16T20:00:32.316ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6872025-01-16T20:00:32.316ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6882025-01-16T20:00:32.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6892025-01-16T20:00:32.316ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 549358813 } })
6902025-01-16T20:00:32.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6912025-01-16T20:00:32.316ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1104 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
6922025-01-16T20:00:32.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6932025-01-16T20:00:32.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6942025-01-16T20:00:32.317ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6952025-01-16T20:00:32.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6962025-01-16T20:00:32.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6972025-01-16T20:00:32.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6982025-01-16T20:00:32.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6992025-01-16T20:00:32.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
7002025-01-16T20:00:32.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7012025-01-16T20:00:32.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7022025-01-16T20:00:32.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7032025-01-16T20:00:32.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7042025-01-16T20:00:32.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7052025-01-16T20:00:32.332ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
7062025-01-16T20:00:32.332ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.63372ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
7072025-01-16T20:00:32.343ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7082025-01-16T20:00:32.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7092025-01-16T20:00:32.344ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7102025-01-16T20:00:32.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7112025-01-16T20:00:32.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 51.25617ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7122025-01-16T20:00:32.352ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7132025-01-16T20:00:32.352ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 53.505648ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
7142025-01-16T20:00:32.352ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 54.15416ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7152025-01-16T20:00:32.360ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7162025-01-16T20:00:32.360ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7172025-01-16T20:00:32.360ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 61.57459ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7182025-01-16T20:00:32.361ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7192025-01-16T20:00:32.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7202025-01-16T20:00:32.412ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
7212025-01-16T20:00:32.412ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7222025-01-16T20:00:32.413ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "862ee236-2de7-4548-b337-d36c2fd2a7d3", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7232025-01-16T20:00:32.413ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7242025-01-16T20:00:32.415ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7252025-01-16T20:00:32.418ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
7262025-01-16T20:00:32.418ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7272025-01-16T20:00:32.418ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "244dfe44-b269-4d98-95f2-cc7dbf047c58", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7282025-01-16T20:00:32.418ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7292025-01-16T20:00:32.420ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7302025-01-16T20:00:32.423ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 123.951145ms iteration = 27 status = {"physical_disks_added":0}
7312025-01-16T20:00:32.425ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7322025-01-16T20:00:32.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7332025-01-16T20:00:32.427ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7342025-01-16T20:00:32.427ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7352025-01-16T20:00:32.427ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "39bb6002-a4d9-45d3-81ff-0e198f81fd1a", "content-length": "372", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7362025-01-16T20:00:32.427ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7372025-01-16T20:00:32.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7382025-01-16T20:00:32.429ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7392025-01-16T20:00:32.429ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7402025-01-16T20:00:32.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7412025-01-16T20:00:32.430ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 130.030611ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7422025-01-16T20:00:32.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7432025-01-16T20:00:32.431ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7442025-01-16T20:00:32.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7452025-01-16T20:00:32.432ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1112 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f zone_address = fd00:1122:3344:101::2
7462025-01-16T20:00:32.432ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7472025-01-16T20:00:32.433ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c15b6a40-0d48-4ef2-a17f-68590534a476", "content-length": "26", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7482025-01-16T20:00:32.433ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1117 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7492025-01-16T20:00:32.435ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
7502025-01-16T20:00:32.460ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7512025-01-16T20:00:32.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7522025-01-16T20:00:32.462ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7532025-01-16T20:00:32.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7542025-01-16T20:00:32.462ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:370
7552025-01-16T20:00:32.462ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 163.830245ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7562025-01-16T20:00:32.467ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7572025-01-16T20:00:32.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87207105-a825-4eaa-ac6d-6aaa586b0fc5, lookup_type: ById(87207105-a825-4eaa-ac6d-6aaa586b0fc5) } result = Ok(())
7582025-01-16T20:00:32.468ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7592025-01-16T20:00:32.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7602025-01-16T20:00:32.470ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 171.576451ms iteration = 24 status = 0
7612025-01-16T20:00:32.482ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7622025-01-16T20:00:32.482ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7632025-01-16T20:00:32.483ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1e246e65-6550-4c77-b847-2b3ba081a24b", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7642025-01-16T20:00:32.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7652025-01-16T20:00:32.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7662025-01-16T20:00:32.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7672025-01-16T20:00:32.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7682025-01-16T20:00:32.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7692025-01-16T20:00:32.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7702025-01-16T20:00:32.492ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7712025-01-16T20:00:32.492ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7722025-01-16T20:00:32.492ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "344d11fa-b75c-40ec-b370-2acd1ccb35c0", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7732025-01-16T20:00:32.493ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 194.85527ms iteration = 25 status = {}
7742025-01-16T20:00:32.496ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 197.348243ms iteration = 24 status = {}
7752025-01-16T20:00:32.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7762025-01-16T20:00:32.500ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7772025-01-16T20:00:32.500ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7782025-01-16T20:00:32.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7792025-01-16T20:00:32.507ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7802025-01-16T20:00:32.507ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cdbaff6-d316-4878-927b-c74f6a22169e", "content-length": "2", "date": "Thu, 16 Jan 2025 20:00:31 GMT"} })
7812025-01-16T20:00:32.510ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7822025-01-16T20:00:32.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7832025-01-16T20:00:32.512ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 213.163341ms iteration = 24 status = null
7842025-01-16T20:00:32.587ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7852025-01-16T20:00:32.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7862025-01-16T20:00:32.601ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7872025-01-16T20:00:32.602ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4f55050-a240-4f56-82c6-d1585f6ca556", "content-length": "691", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
7882025-01-16T20:00:32.602ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
7892025-01-16T20:00:32.602ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
7902025-01-16T20:00:32.602ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
7912025-01-16T20:00:32.603ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 304.877ms iteration = 25 status = {}
7922025-01-16T20:00:32.603ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6ec20bf0-03fa-4e36-9de7-54dd5e9cfde1, rack_id: 26dc04a7-8537-47d8-965d-0931849f651f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(438946af-d2bd-4eab-8036-57fe8293b9f9) } rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
7932025-01-16T20:00:32.603ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7942025-01-16T20:00:32.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7952025-01-16T20:00:32.612ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7962025-01-16T20:00:32.612ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "21bcfc89-51d8-4a0f-9fda-97c30e266375", "content-length": "134", "date": "Thu, 16 Jan 2025 20:00:31 GMT"} })
7972025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7982025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 438946af-d2bd-4eab-8036-57fe8293b9f9
7992025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8002025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8012025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8022025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8032025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8042025-01-16T20:00:32.612ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8052025-01-16T20:00:32.613ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8062025-01-16T20:00:32.613ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8072025-01-16T20:00:32.613ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
8082025-01-16T20:00:32.613ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8092025-01-16T20:00:32.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8102025-01-16T20:00:32.615ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8112025-01-16T20:00:32.616ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "7df3943a-d042-4b88-8b34-c4fa12b3df83", "content-length": "2", "date": "Thu, 16 Jan 2025 20:00:31 GMT"} })
8122025-01-16T20:00:32.616ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8132025-01-16T20:00:32.616ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a40ee10e-48f9-4783-857f-9fb6ae6364ad", "content-length": "52", "date": "Thu, 16 Jan 2025 20:00:31 GMT"} })
8142025-01-16T20:00:32.616ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
8152025-01-16T20:00:32.616ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8162025-01-16T20:00:32.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 620 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 80cbd156-6904-4d5d-a844-e6797b5f9215 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
8172025-01-16T20:00:32.740ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "24303086-894f-47bc-ad65-d35354d05c07", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
8182025-01-16T20:00:32.740ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch = Switch0
8192025-01-16T20:00:32.740ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8202025-01-16T20:00:32.740ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3911551-408e-4089-a834-43d12e770a56", "content-length": "682", "date": "Thu, 16 Jan 2025 20:00:32 GMT"} })
8212025-01-16T20:00:32.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8222025-01-16T20:00:32.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8232025-01-16T20:00:32.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8242025-01-16T20:00:32.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8252025-01-16T20:00:32.750ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8262025-01-16T20:00:32.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8272025-01-16T20:00:32.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8282025-01-16T20:00:32.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8292025-01-16T20:00:32.761ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 462.685189ms iteration = 24 status = {}
8302025-01-16T20:00:33.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 154a117b-4e05-4ea1-a667-13cf6d31eca5 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8312025-01-16T20:00:33.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 154a117b-4e05-4ea1-a667-13cf6d31eca5 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8322025-01-16T20:00:33.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2416 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 154a117b-4e05-4ea1-a667-13cf6d31eca5 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8332025-01-16T20:00:34.277ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 96c52141-445b-4997-8e9e-bd24bb483200 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8342025-01-16T20:00:34.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 96c52141-445b-4997-8e9e-bd24bb483200 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8352025-01-16T20:00:34.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2651 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 96c52141-445b-4997-8e9e-bd24bb483200 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8362025-01-16T20:00:35.299ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 6865cd97-780f-468e-a560-9f84d6a850fa roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8372025-01-16T20:00:35.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 6865cd97-780f-468e-a560-9f84d6a850fa resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8382025-01-16T20:00:35.301ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2392 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 6865cd97-780f-468e-a560-9f84d6a850fa response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8392025-01-16T20:00:36.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8869a1c2-3a1b-46b7-a193-bf6a058c4626 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8402025-01-16T20:00:36.320ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8869a1c2-3a1b-46b7-a193-bf6a058c4626 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8412025-01-16T20:00:36.322ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2516 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8869a1c2-3a1b-46b7-a193-bf6a058c4626 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8422025-01-16T20:00:37.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 78d7f9dc-ffc0-4a51-8e0d-a473a8d643de roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8432025-01-16T20:00:37.341ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 78d7f9dc-ffc0-4a51-8e0d-a473a8d643de resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8442025-01-16T20:00:37.343ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2884 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 78d7f9dc-ffc0-4a51-8e0d-a473a8d643de response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8452025-01-16T20:00:38.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 88140cb5-d9da-4ac9-b14f-b357b98d43e0 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8462025-01-16T20:00:38.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 88140cb5-d9da-4ac9-b14f-b357b98d43e0 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8472025-01-16T20:00:38.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2099 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 88140cb5-d9da-4ac9-b14f-b357b98d43e0 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8482025-01-16T20:00:39.383ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5d47267c-1f9a-4f1e-84de-9c1dd368b503 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8492025-01-16T20:00:39.383ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5d47267c-1f9a-4f1e-84de-9c1dd368b503 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8502025-01-16T20:00:39.385ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2642 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5d47267c-1f9a-4f1e-84de-9c1dd368b503 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8512025-01-16T20:00:40.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 6b44ebeb-ea95-4d2c-b87f-57edcbe5ebe3 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8522025-01-16T20:00:40.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 6b44ebeb-ea95-4d2c-b87f-57edcbe5ebe3 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8532025-01-16T20:00:40.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2237 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 6b44ebeb-ea95-4d2c-b87f-57edcbe5ebe3 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8542025-01-16T20:00:41.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = bb021e4c-643d-42d6-81d8-f1f963d62bba roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8552025-01-16T20:00:41.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = bb021e4c-643d-42d6-81d8-f1f963d62bba resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8562025-01-16T20:00:41.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1955 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = bb021e4c-643d-42d6-81d8-f1f963d62bba response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8572025-01-16T20:00:42.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
8582025-01-16T20:00:42.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
8592025-01-16T20:00:42.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8602025-01-16T20:00:42.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8612025-01-16T20:00:42.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8622025-01-16T20:00:42.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8632025-01-16T20:00:42.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8642025-01-16T20:00:42.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8652025-01-16T20:00:42.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8662025-01-16T20:00:42.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8672025-01-16T20:00:42.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8682025-01-16T20:00:42.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8692025-01-16T20:00:42.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8702025-01-16T20:00:42.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
8712025-01-16T20:00:42.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8722025-01-16T20:00:42.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8732025-01-16T20:00:42.323ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
8742025-01-16T20:00:42.323ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.617574ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
8752025-01-16T20:00:42.329ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8762025-01-16T20:00:42.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8772025-01-16T20:00:42.331ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.698086ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8782025-01-16T20:00:42.448ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1ee87887-5a1f-4ee7-9b06-389670bd756c roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8792025-01-16T20:00:42.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1ee87887-5a1f-4ee7-9b06-389670bd756c resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8802025-01-16T20:00:42.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2036 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1ee87887-5a1f-4ee7-9b06-389670bd756c response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8812025-01-16T20:00:42.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 07739155-1826-4195-85ce-ebc5eb4bf576 roles = RoleSet { roles: {} } uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
8822025-01-16T20:00:42.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 07739155-1826-4195-85ce-ebc5eb4bf576 resource = Database result = Ok(()) uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
8832025-01-16T20:00:42.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2690 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 07739155-1826-4195-85ce-ebc5eb4bf576 response_code = 200 uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
8842025-01-16T20:00:42.674ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 593847bc-a7a9-4846-a1e3-ae769a475c1b roles = RoleSet { roles: {} } uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
8852025-01-16T20:00:42.674ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 593847bc-a7a9-4846-a1e3-ae769a475c1b resource = Database result = Ok(()) uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
8862025-01-16T20:00:42.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2069 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 593847bc-a7a9-4846-a1e3-ae769a475c1b response_code = 200 uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
8872025-01-16T20:00:42.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 574 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 6d25971e-ed0b-4ada-93e5-3f79e70851a5 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
8882025-01-16T20:00:43.470ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3c7fe152-8c59-49ea-8d94-611259a63f29 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8892025-01-16T20:00:43.470ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3c7fe152-8c59-49ea-8d94-611259a63f29 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8902025-01-16T20:00:43.472ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2283 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3c7fe152-8c59-49ea-8d94-611259a63f29 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8912025-01-16T20:00:44.491ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0b6807f7-de12-455f-97b6-a91f205956e1 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8922025-01-16T20:00:44.492ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0b6807f7-de12-455f-97b6-a91f205956e1 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8932025-01-16T20:00:44.494ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2361 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0b6807f7-de12-455f-97b6-a91f205956e1 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8942025-01-16T20:00:45.123ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 11b043c0-bd14-4b7b-aebb-8556f8192c35 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
8952025-01-16T20:00:45.124ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 11b043c0-bd14-4b7b-aebb-8556f8192c35 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
8962025-01-16T20:00:45.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3683 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 11b043c0-bd14-4b7b-aebb-8556f8192c35 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
8972025-01-16T20:00:45.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8f4b1ea6-94d4-4995-ae68-3c56887be0f9 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8982025-01-16T20:00:45.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8f4b1ea6-94d4-4995-ae68-3c56887be0f9 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
8992025-01-16T20:00:45.515ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2474 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8f4b1ea6-94d4-4995-ae68-3c56887be0f9 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9002025-01-16T20:00:46.533ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 65c36cb7-b9ad-42f7-9780-98c57697819e roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9012025-01-16T20:00:46.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 65c36cb7-b9ad-42f7-9780-98c57697819e resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9022025-01-16T20:00:46.536ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2366 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 65c36cb7-b9ad-42f7-9780-98c57697819e response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9032025-01-16T20:00:47.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0090f6ce-a7b3-4f0d-bb26-ab0747dc4c52 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9042025-01-16T20:00:47.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0090f6ce-a7b3-4f0d-bb26-ab0747dc4c52 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9052025-01-16T20:00:47.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0090f6ce-a7b3-4f0d-bb26-ab0747dc4c52 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9062025-01-16T20:00:48.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0d247571-c8a3-4a1e-b87e-8bbe085bcc43 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9072025-01-16T20:00:48.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0d247571-c8a3-4a1e-b87e-8bbe085bcc43 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9082025-01-16T20:00:48.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2257 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0d247571-c8a3-4a1e-b87e-8bbe085bcc43 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9092025-01-16T20:00:49.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5e0e19d2-dbb3-4218-a912-2d08bf19be3c roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9102025-01-16T20:00:49.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5e0e19d2-dbb3-4218-a912-2d08bf19be3c resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9112025-01-16T20:00:49.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2417 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5e0e19d2-dbb3-4218-a912-2d08bf19be3c response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9122025-01-16T20:00:50.619ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e1267b62-abab-4a4b-8e7a-b81dc62dba4d roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9132025-01-16T20:00:50.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e1267b62-abab-4a4b-8e7a-b81dc62dba4d resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9142025-01-16T20:00:50.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2333 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e1267b62-abab-4a4b-8e7a-b81dc62dba4d response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9152025-01-16T20:00:51.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f9e466bb-bc23-4e29-9718-e2aa4f639afe roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9162025-01-16T20:00:51.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f9e466bb-bc23-4e29-9718-e2aa4f639afe resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9172025-01-16T20:00:51.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2656 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f9e466bb-bc23-4e29-9718-e2aa4f639afe response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9182025-01-16T20:00:52.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
9192025-01-16T20:00:52.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9202025-01-16T20:00:52.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
9212025-01-16T20:00:52.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9222025-01-16T20:00:52.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9232025-01-16T20:00:52.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9242025-01-16T20:00:52.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9252025-01-16T20:00:52.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9262025-01-16T20:00:52.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9272025-01-16T20:00:52.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9282025-01-16T20:00:52.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9292025-01-16T20:00:52.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9302025-01-16T20:00:52.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9312025-01-16T20:00:52.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
9322025-01-16T20:00:52.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9332025-01-16T20:00:52.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9342025-01-16T20:00:52.323ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
9352025-01-16T20:00:52.323ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.189535ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
9362025-01-16T20:00:52.329ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9372025-01-16T20:00:52.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9382025-01-16T20:00:52.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.13381ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9392025-01-16T20:00:52.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 245b7e42-853e-4360-ae66-b0d847a662d4 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9402025-01-16T20:00:52.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 245b7e42-853e-4360-ae66-b0d847a662d4 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9412025-01-16T20:00:52.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2108 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 245b7e42-853e-4360-ae66-b0d847a662d4 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9422025-01-16T20:00:52.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 535 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 70a74e2c-0085-427f-8153-2cad56392524 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
9432025-01-16T20:00:53.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5d7fb60c-52ca-45f2-9424-07952c7cd5cc roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9442025-01-16T20:00:53.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5d7fb60c-52ca-45f2-9424-07952c7cd5cc resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9452025-01-16T20:00:53.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2208 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5d7fb60c-52ca-45f2-9424-07952c7cd5cc response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9462025-01-16T20:00:54.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 359c44a1-78c9-4d3f-bc45-ee9a9cf644ce roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9472025-01-16T20:00:54.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 359c44a1-78c9-4d3f-bc45-ee9a9cf644ce resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9482025-01-16T20:00:54.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2107 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 359c44a1-78c9-4d3f-bc45-ee9a9cf644ce response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9492025-01-16T20:00:55.725ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 982593a4-1be9-4364-9cfb-b2647c5050c6 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9502025-01-16T20:00:55.726ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 982593a4-1be9-4364-9cfb-b2647c5050c6 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9512025-01-16T20:00:55.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2220 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 982593a4-1be9-4364-9cfb-b2647c5050c6 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9522025-01-16T20:00:56.747ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a8cb85b4-aab7-499e-aeb4-bb9122801166 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9532025-01-16T20:00:56.747ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a8cb85b4-aab7-499e-aeb4-bb9122801166 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9542025-01-16T20:00:56.749ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2633 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a8cb85b4-aab7-499e-aeb4-bb9122801166 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9552025-01-16T20:00:57.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5a00513a-1d80-4276-8576-ed8ebee29768 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9562025-01-16T20:00:57.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5a00513a-1d80-4276-8576-ed8ebee29768 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9572025-01-16T20:00:57.769ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1925 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5a00513a-1d80-4276-8576-ed8ebee29768 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9582025-01-16T20:00:58.789ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 34ab18c0-b401-429f-959d-bfdda22ba7b2 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9592025-01-16T20:00:58.789ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 34ab18c0-b401-429f-959d-bfdda22ba7b2 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9602025-01-16T20:00:58.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2225 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 34ab18c0-b401-429f-959d-bfdda22ba7b2 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9612025-01-16T20:00:59.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e58067c0-e2b5-457c-9139-c629e051c99f roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9622025-01-16T20:00:59.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e58067c0-e2b5-457c-9139-c629e051c99f resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9632025-01-16T20:00:59.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2253 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e58067c0-e2b5-457c-9139-c629e051c99f response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9642025-01-16T20:01:00.831ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b613fb08-039a-44ad-8943-63fc7182c071 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9652025-01-16T20:01:00.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b613fb08-039a-44ad-8943-63fc7182c071 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9662025-01-16T20:01:00.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3019 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b613fb08-039a-44ad-8943-63fc7182c071 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9672025-01-16T20:01:01.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3b541b7e-96b3-43ee-8017-262b5e794afe roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9682025-01-16T20:01:01.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3b541b7e-96b3-43ee-8017-262b5e794afe resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9692025-01-16T20:01:01.858ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2001 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3b541b7e-96b3-43ee-8017-262b5e794afe response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
9702025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
9712025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
9722025-01-16T20:01:02.299ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
9732025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
9742025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9752025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
9762025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
9772025-01-16T20:01:02.299ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-16 19:51:02.299822667 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9782025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9792025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
9802025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9812025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
9822025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9832025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
9842025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
9852025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9862025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9872025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
9882025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9892025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9902025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
9912025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
9922025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9932025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
9942025-01-16T20:01:02.299ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9952025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
9962025-01-16T20:01:02.300ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8
9972025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9982025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
9992025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.321\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8"}
10002025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
10012025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10022025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
10032025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10042025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10052025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
10062025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
10072025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10082025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
10092025-01-16T20:01:02.300ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10102025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10112025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
10122025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10132025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10142025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
10152025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
10162025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10172025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10182025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
10192025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10202025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10212025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
10222025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
10232025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10242025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
10252025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10262025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
10272025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10282025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
10292025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10302025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
10312025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
10322025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10332025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10342025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
10352025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10362025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10372025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10382025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10392025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10402025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10412025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10422025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10432025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10442025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10452025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10462025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10472025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10482025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10492025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10502025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10512025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10522025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10532025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10542025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
10552025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10562025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10572025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10582025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10592025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10602025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10612025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10622025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10632025-01-16T20:01:02.300ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1673b82a-d301-40a0-97f5-01c1919bcccc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a78ec168-75b0-4c15-b272-0f34e450b715.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fc5e4155-1b26-42b3-8db4-a4cf6dd61fe9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fc5e4155-1b26-42b3-8db4-a4cf6dd61fe9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 533421694 } })
10642025-01-16T20:01:02.300ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("67baed66-640b-413d-8436-22e580c71fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b6525081-e08a-4759-918e-50f60ff0f6f1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b6525081-e08a-4759-918e-50f60ff0f6f1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 533447874 } })
10652025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10662025-01-16T20:01:02.300ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10672025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10682025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10692025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10702025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10712025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10722025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.344884ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
10732025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
10742025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.621849ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
10752025-01-16T20:01:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10762025-01-16T20:01:02.302ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10772025-01-16T20:01:02.303ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 536211472 } })
10782025-01-16T20:01:02.303ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.577823ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10792025-01-16T20:01:02.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10802025-01-16T20:01:02.303ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1104
10812025-01-16T20:01:02.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
10822025-01-16T20:01:02.304ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10832025-01-16T20:01:02.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10842025-01-16T20:01:02.305ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-16 19:51:02.299822667 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10852025-01-16T20:01:02.305ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.237622ms iteration = 13 status = {"expiration":"2025-01-16T19:51:02.299822667Z","pruned":[]}
10862025-01-16T20:01:02.306ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872025-01-16T20:01:02.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10882025-01-16T20:01:02.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10892025-01-16T20:01:02.307ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10902025-01-16T20:01:02.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10912025-01-16T20:01:02.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10922025-01-16T20:01:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10932025-01-16T20:01:02.308ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10942025-01-16T20:01:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10952025-01-16T20:01:02.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10962025-01-16T20:01:02.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10972025-01-16T20:01:02.310ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 543127494 } })
10982025-01-16T20:01:02.310ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1104
10992025-01-16T20:01:02.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11002025-01-16T20:01:02.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11012025-01-16T20:01:02.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11022025-01-16T20:01:02.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11032025-01-16T20:01:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11042025-01-16T20:01:02.316ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11052025-01-16T20:01:02.316ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
11062025-01-16T20:01:02.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11072025-01-16T20:01:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11082025-01-16T20:01:02.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11092025-01-16T20:01:02.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11102025-01-16T20:01:02.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11112025-01-16T20:01:02.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11122025-01-16T20:01:02.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11132025-01-16T20:01:02.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11142025-01-16T20:01:02.319ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.749077ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
11152025-01-16T20:01:02.320ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11162025-01-16T20:01:02.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11172025-01-16T20:01:02.322ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11182025-01-16T20:01:02.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11192025-01-16T20:01:02.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11202025-01-16T20:01:02.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11212025-01-16T20:01:02.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11222025-01-16T20:01:02.326ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11232025-01-16T20:01:02.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11242025-01-16T20:01:02.327ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11252025-01-16T20:01:02.327ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11262025-01-16T20:01:02.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11272025-01-16T20:01:02.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11282025-01-16T20:01:02.329ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11292025-01-16T20:01:02.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11302025-01-16T20:01:02.331ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11312025-01-16T20:01:02.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11322025-01-16T20:01:02.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11332025-01-16T20:01:02.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11342025-01-16T20:01:02.333ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11352025-01-16T20:01:02.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11362025-01-16T20:01:02.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11372025-01-16T20:01:02.337ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 569445106 } })
11382025-01-16T20:01:02.337ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1104 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
11392025-01-16T20:01:02.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11402025-01-16T20:01:02.341ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11412025-01-16T20:01:02.345ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11422025-01-16T20:01:02.347ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11432025-01-16T20:01:02.349ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11442025-01-16T20:01:02.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11452025-01-16T20:01:02.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11462025-01-16T20:01:02.350ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11472025-01-16T20:01:02.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11482025-01-16T20:01:02.354ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11492025-01-16T20:01:02.354ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11502025-01-16T20:01:02.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11512025-01-16T20:01:02.355ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.392483ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11522025-01-16T20:01:02.356ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11532025-01-16T20:01:02.356ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11542025-01-16T20:01:02.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11552025-01-16T20:01:02.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11562025-01-16T20:01:02.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
11572025-01-16T20:01:02.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11582025-01-16T20:01:02.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11592025-01-16T20:01:02.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11602025-01-16T20:01:02.368ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11612025-01-16T20:01:02.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11622025-01-16T20:01:02.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11632025-01-16T20:01:02.369ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11642025-01-16T20:01:02.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11652025-01-16T20:01:02.371ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11662025-01-16T20:01:02.371ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11672025-01-16T20:01:02.371ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11682025-01-16T20:01:02.373ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11692025-01-16T20:01:02.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11702025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11712025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11722025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.258343ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
11732025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11742025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11752025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11762025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772025-01-16T20:01:02.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11782025-01-16T20:01:02.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11792025-01-16T20:01:02.375ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11802025-01-16T20:01:02.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11812025-01-16T20:01:02.376ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11822025-01-16T20:01:02.377ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11832025-01-16T20:01:02.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11842025-01-16T20:01:02.377ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
11852025-01-16T20:01:02.377ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
11862025-01-16T20:01:02.377ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 77.118833ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
11872025-01-16T20:01:02.378ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11882025-01-16T20:01:02.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11892025-01-16T20:01:02.378ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 78.750222ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11902025-01-16T20:01:02.379ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.302991ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
11912025-01-16T20:01:02.380ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11922025-01-16T20:01:02.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11932025-01-16T20:01:02.380ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11942025-01-16T20:01:02.381ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11952025-01-16T20:01:02.382ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11962025-01-16T20:01:02.382ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11972025-01-16T20:01:02.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11982025-01-16T20:01:02.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11992025-01-16T20:01:02.383ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12002025-01-16T20:01:02.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12012025-01-16T20:01:02.384ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 84.35371ms iteration = 6 status = {"nsuccess":5}
12022025-01-16T20:01:02.385ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12032025-01-16T20:01:02.389ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12042025-01-16T20:01:02.389ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12052025-01-16T20:01:02.389ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12062025-01-16T20:01:02.389ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12072025-01-16T20:01:02.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12082025-01-16T20:01:02.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12092025-01-16T20:01:02.390ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12102025-01-16T20:01:02.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12112025-01-16T20:01:02.392ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 92.383691ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12122025-01-16T20:01:02.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12132025-01-16T20:01:02.393ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12142025-01-16T20:01:02.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12152025-01-16T20:01:02.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12162025-01-16T20:01:02.393ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
12172025-01-16T20:01:02.393ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.574812ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
12182025-01-16T20:01:02.395ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12192025-01-16T20:01:02.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12202025-01-16T20:01:02.397ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
12212025-01-16T20:01:02.397ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
12222025-01-16T20:01:02.397ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 98.086122ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
12232025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12242025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12252025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12262025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12272025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12282025-01-16T20:01:02.398ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12292025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
12302025-01-16T20:01:02.398ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12312025-01-16T20:01:02.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12322025-01-16T20:01:02.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12332025-01-16T20:01:02.400ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12342025-01-16T20:01:02.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87207105-a825-4eaa-ac6d-6aaa586b0fc5, lookup_type: ById(87207105-a825-4eaa-ac6d-6aaa586b0fc5) } result = Ok(())
12352025-01-16T20:01:02.401ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
12362025-01-16T20:01:02.401ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12372025-01-16T20:01:02.401ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12382025-01-16T20:01:02.401ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 101.592213ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
12392025-01-16T20:01:02.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12402025-01-16T20:01:02.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
12412025-01-16T20:01:02.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12422025-01-16T20:01:02.403ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12432025-01-16T20:01:02.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12442025-01-16T20:01:02.403ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 103.807341ms iteration = 25 status = 0
12452025-01-16T20:01:02.418ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12462025-01-16T20:01:02.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12472025-01-16T20:01:02.419ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12482025-01-16T20:01:02.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12492025-01-16T20:01:02.420ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12502025-01-16T20:01:02.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12512025-01-16T20:01:02.422ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12522025-01-16T20:01:02.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12532025-01-16T20:01:02.431ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12542025-01-16T20:01:02.433ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12552025-01-16T20:01:02.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12562025-01-16T20:01:02.434ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12572025-01-16T20:01:02.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12582025-01-16T20:01:02.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12592025-01-16T20:01:02.435ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12602025-01-16T20:01:02.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12612025-01-16T20:01:02.436ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12622025-01-16T20:01:02.436ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 481516d2-3f4d-4dc5-9036-8c14a1fa4fd7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12632025-01-16T20:01:02.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12642025-01-16T20:01:02.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12652025-01-16T20:01:02.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12662025-01-16T20:01:02.441ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12672025-01-16T20:01:02.441ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12682025-01-16T20:01:02.441ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 141.723767ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12692025-01-16T20:01:02.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12702025-01-16T20:01:02.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12712025-01-16T20:01:02.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12722025-01-16T20:01:02.448ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 481516d2-3f4d-4dc5-9036-8c14a1fa4fd7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
12732025-01-16T20:01:02.448ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
12742025-01-16T20:01:02.448ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-16 19:49:07.686045 UTC dns_group = internal generation = 1
12752025-01-16T20:01:02.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 148.454405ms iteration = 13 status = {"generation":1}
12762025-01-16T20:01:02.451ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 150.943238ms iteration = 28 status = {"physical_disks_added":0}
12772025-01-16T20:01:02.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 154.898039ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
12782025-01-16T20:01:02.460ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12792025-01-16T20:01:02.462ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12802025-01-16T20:01:02.466ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:370
12812025-01-16T20:01:02.466ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 166.206637ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
12822025-01-16T20:01:02.474ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12832025-01-16T20:01:02.474ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = f05be43b-5d84-4b95-99be-d6d0a1e3a923 dns_zone_name = oxide.test found_so_far = 0 version = 2
12842025-01-16T20:01:02.474ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12852025-01-16T20:01:02.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12862025-01-16T20:01:02.482ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-16T19:48:57.779461Z, time_modified: 2025-01-16T19:48:57.779461Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 197fa9bd-2b54-417a-8249-a5354e078485, db_silo: Silo { identity: SiloIdentity { id: 197fa9bd-2b54-417a-8249-a5354e078485, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-16T19:49:10.527303Z, time_modified: 2025-01-16T19:49:10.527303Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 197fa9bd-2b54-417a-8249-a5354e078485, db_silo: Silo { identity: SiloIdentity { id: 197fa9bd-2b54-417a-8249-a5354e078485, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-16T19:49:10.527303Z, time_modified: 2025-01-16T19:49:10.527303Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538" }] }) }
12872025-01-16T20:01:02.482ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 182.984411ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"197fa9bd-2b54-417a-8249-a5354e078485","tls_certs":["b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538"]}},"default_endpoint":{"silo_id":"197fa9bd-2b54-417a-8249-a5354e078485","tls_certs":["b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
12882025-01-16T20:01:02.510ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
12892025-01-16T20:01:02.511ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 211.020653ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
12902025-01-16T20:01:02.530ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1112 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f zone_address = fd00:1122:3344:101::2
12912025-01-16T20:01:02.530ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12922025-01-16T20:01:02.530ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "12a9a832-abdd-4b4a-a600-087832a6fcfe", "content-length": "26", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
12932025-01-16T20:01:02.530ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1117 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12942025-01-16T20:01:02.533ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
12952025-01-16T20:01:02.567ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
12962025-01-16T20:01:02.567ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12972025-01-16T20:01:02.568ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9194e635-443f-49b4-8646-8f61e9051a85", "content-length": "26", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
12982025-01-16T20:01:02.568ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12992025-01-16T20:01:02.569ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13002025-01-16T20:01:02.604ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
13012025-01-16T20:01:02.604ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13022025-01-16T20:01:02.604ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f141db15-9b53-4f50-a535-8eac3f5dde22", "content-length": "26", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13032025-01-16T20:01:02.604ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13042025-01-16T20:01:02.607ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13052025-01-16T20:01:02.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 732 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 94b81343-98f4-43d9-95a3-085ace36f533 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
13062025-01-16T20:01:02.818ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13072025-01-16T20:01:02.819ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "08824ef9-dd40-4578-bbb7-50842735ca6c", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13082025-01-16T20:01:02.819ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13092025-01-16T20:01:02.820ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c1b412e9-75d7-4d77-acc1-fc4454b0439b", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13102025-01-16T20:01:02.820ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13112025-01-16T20:01:02.822ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13122025-01-16T20:01:02.822ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 522.993887ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
13132025-01-16T20:01:02.826ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13142025-01-16T20:01:02.826ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a4e92cf-112a-496b-9e18-5a358fb2253c", "content-length": "691", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13152025-01-16T20:01:02.826ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
13162025-01-16T20:01:02.826ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
13172025-01-16T20:01:02.826ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
13182025-01-16T20:01:02.828ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 528.33104ms iteration = 26 status = {}
13192025-01-16T20:01:02.843ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 543.798202ms iteration = 25 status = {}
13202025-01-16T20:01:02.854ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13212025-01-16T20:01:02.855ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a42d9d2-0d31-4656-b806-5ea685c29f68", "content-length": "2", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13222025-01-16T20:01:02.855ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13232025-01-16T20:01:02.857ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13242025-01-16T20:01:02.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13252025-01-16T20:01:02.869ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13262025-01-16T20:01:02.869ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13272025-01-16T20:01:02.869ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "18efc0a3-3133-4216-9d76-04537dbbb2db", "content-length": "372", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13282025-01-16T20:01:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13292025-01-16T20:01:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13302025-01-16T20:01:02.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13312025-01-16T20:01:02.872ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 572.537156ms iteration = 25 status = null
13322025-01-16T20:01:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13332025-01-16T20:01:02.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13342025-01-16T20:01:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13352025-01-16T20:01:02.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f88d2d0e-529d-420c-8e67-2fdbc9ea3833 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
13362025-01-16T20:01:02.879ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = f05be43b-5d84-4b95-99be-d6d0a1e3a923 dns_zone_name = oxide.test found_so_far = 1 version = 2
13372025-01-16T20:01:02.879ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13382025-01-16T20:01:02.879ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-16 19:49:10.849798 UTC dns_group = external generation = 2
13392025-01-16T20:01:02.879ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 579.273495ms iteration = 13 status = {"generation":2}
13402025-01-16T20:01:02.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f88d2d0e-529d-420c-8e67-2fdbc9ea3833 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
13412025-01-16T20:01:02.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2018 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f88d2d0e-529d-420c-8e67-2fdbc9ea3833 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
13422025-01-16T20:01:02.947ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
13432025-01-16T20:01:02.947ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13442025-01-16T20:01:02.947ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d15d12bc-e68e-4346-87cb-7bfc9558a839", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13452025-01-16T20:01:02.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13462025-01-16T20:01:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13472025-01-16T20:01:02.949ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13482025-01-16T20:01:02.950ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13492025-01-16T20:01:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13502025-01-16T20:01:02.950ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f174efbb-3019-4834-8c7d-a6221afcb525", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13512025-01-16T20:01:02.950ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13522025-01-16T20:01:02.952ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "210463ee-6506-422c-b2df-170732c43224", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13532025-01-16T20:01:02.952ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13542025-01-16T20:01:02.953ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13552025-01-16T20:01:02.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13562025-01-16T20:01:02.954ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c77216d8-fc8b-4c81-9685-c3ea3237a9bb", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13572025-01-16T20:01:02.954ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13582025-01-16T20:01:02.957ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13592025-01-16T20:01:02.957ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 657.435016ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13602025-01-16T20:01:02.957ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13612025-01-16T20:01:02.957ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13622025-01-16T20:01:02.958ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e9721627-e748-4d78-be25-99f48e58e7e6", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13632025-01-16T20:01:02.960ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 660.060202ms iteration = 26 status = {}
13642025-01-16T20:01:03.002ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13652025-01-16T20:01:03.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13662025-01-16T20:01:03.005ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6ec20bf0-03fa-4e36-9de7-54dd5e9cfde1, rack_id: 26dc04a7-8537-47d8-965d-0931849f651f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(438946af-d2bd-4eab-8036-57fe8293b9f9) } rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
13672025-01-16T20:01:03.005ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13682025-01-16T20:01:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13692025-01-16T20:01:03.013ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13702025-01-16T20:01:03.014ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "de3b9445-c413-4cc9-a62f-59f85de1cbc2", "content-length": "134", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13712025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
13722025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 438946af-d2bd-4eab-8036-57fe8293b9f9
13732025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
13742025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13752025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13762025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13772025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13782025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13792025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13802025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13812025-01-16T20:01:03.014ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
13822025-01-16T20:01:03.014ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13832025-01-16T20:01:03.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13842025-01-16T20:01:03.017ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13852025-01-16T20:01:03.018ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "91d02c47-d91e-4617-aba6-f71f6b1f4057", "content-length": "2", "date": "Thu, 16 Jan 2025 20:01:03 GMT"} })
13862025-01-16T20:01:03.018ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13872025-01-16T20:01:03.018ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "244de51e-fd4b-4f16-8c5c-131c0f0ae5c1", "content-length": "52", "date": "Thu, 16 Jan 2025 20:01:03 GMT"} })
13882025-01-16T20:01:03.018ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
13892025-01-16T20:01:03.018ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13902025-01-16T20:01:03.123ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "77dbc58d-b3f5-49f6-ba82-2bbc1f449586", "date": "Thu, 16 Jan 2025 20:01:02 GMT"} })
13912025-01-16T20:01:03.123ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch = Switch0
13922025-01-16T20:01:03.123ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13932025-01-16T20:01:03.124ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "837e0243-f9eb-4d59-8a54-6fdb42417053", "content-length": "682", "date": "Thu, 16 Jan 2025 20:01:03 GMT"} })
13942025-01-16T20:01:03.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13952025-01-16T20:01:03.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13962025-01-16T20:01:03.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13972025-01-16T20:01:03.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13982025-01-16T20:01:03.131ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13992025-01-16T20:01:03.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14002025-01-16T20:01:03.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14012025-01-16T20:01:03.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14022025-01-16T20:01:03.141ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 841.390864ms iteration = 25 status = {}
14032025-01-16T20:01:03.905ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d9a83f22-8d53-4014-bbbb-e4aec4e5ab23 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14042025-01-16T20:01:03.906ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d9a83f22-8d53-4014-bbbb-e4aec4e5ab23 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14052025-01-16T20:01:03.908ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2079 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d9a83f22-8d53-4014-bbbb-e4aec4e5ab23 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14062025-01-16T20:01:04.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = acc01985-50f8-4f74-b217-3d111428ca8e roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14072025-01-16T20:01:04.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = acc01985-50f8-4f74-b217-3d111428ca8e resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14082025-01-16T20:01:04.929ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2097 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = acc01985-50f8-4f74-b217-3d111428ca8e response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14092025-01-16T20:01:05.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 95c98f09-05d1-4939-8383-74652fe09f2f roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14102025-01-16T20:01:05.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 95c98f09-05d1-4939-8383-74652fe09f2f resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14112025-01-16T20:01:05.950ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2398 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 95c98f09-05d1-4939-8383-74652fe09f2f response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14122025-01-16T20:01:06.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = dc7b2a3e-16fe-4b09-b9fd-9ab35000b222 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14132025-01-16T20:01:06.969ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = dc7b2a3e-16fe-4b09-b9fd-9ab35000b222 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14142025-01-16T20:01:06.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2330 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = dc7b2a3e-16fe-4b09-b9fd-9ab35000b222 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14152025-01-16T20:01:07.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1597538c-003d-4245-a5bd-65a570cc1f49 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14162025-01-16T20:01:07.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1597538c-003d-4245-a5bd-65a570cc1f49 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14172025-01-16T20:01:07.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2226 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1597538c-003d-4245-a5bd-65a570cc1f49 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14182025-01-16T20:01:09.012ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5436e9b5-6b2e-4224-b0fd-1a121dec46fa roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14192025-01-16T20:01:09.012ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5436e9b5-6b2e-4224-b0fd-1a121dec46fa resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14202025-01-16T20:01:09.015ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2803 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5436e9b5-6b2e-4224-b0fd-1a121dec46fa response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14212025-01-16T20:01:10.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a9037e2f-c8bc-448b-a011-df814e804fd5 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14222025-01-16T20:01:10.035ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a9037e2f-c8bc-448b-a011-df814e804fd5 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14232025-01-16T20:01:10.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2544 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a9037e2f-c8bc-448b-a011-df814e804fd5 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14242025-01-16T20:01:11.056ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 196868c1-af1a-44bf-8414-919a4ccd68f3 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14252025-01-16T20:01:11.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 196868c1-af1a-44bf-8414-919a4ccd68f3 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14262025-01-16T20:01:11.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 196868c1-af1a-44bf-8414-919a4ccd68f3 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14272025-01-16T20:01:12.078ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2dd0d54e-92a9-41cb-b69a-3c3c733ce6b1 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14282025-01-16T20:01:12.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2dd0d54e-92a9-41cb-b69a-3c3c733ce6b1 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14292025-01-16T20:01:12.081ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2416 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2dd0d54e-92a9-41cb-b69a-3c3c733ce6b1 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14302025-01-16T20:01:12.300ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
14312025-01-16T20:01:12.300ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
14322025-01-16T20:01:12.300ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14332025-01-16T20:01:12.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14342025-01-16T20:01:12.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14352025-01-16T20:01:12.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14362025-01-16T20:01:12.302ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14372025-01-16T20:01:12.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14382025-01-16T20:01:12.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14392025-01-16T20:01:12.304ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.866418ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14402025-01-16T20:01:12.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14412025-01-16T20:01:12.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14422025-01-16T20:01:12.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14432025-01-16T20:01:12.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14442025-01-16T20:01:12.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14452025-01-16T20:01:12.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14462025-01-16T20:01:12.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
14472025-01-16T20:01:12.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14482025-01-16T20:01:12.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14492025-01-16T20:01:12.320ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
14502025-01-16T20:01:12.320ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.278866ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
14512025-01-16T20:01:12.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 545 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = fa0c4543-306f-4a11-8ab2-ae1b6091e597 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
14522025-01-16T20:01:13.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c1e5906e-9a10-449b-8b0c-5d442a5e3506 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14532025-01-16T20:01:13.100ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c1e5906e-9a10-449b-8b0c-5d442a5e3506 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14542025-01-16T20:01:13.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2046 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c1e5906e-9a10-449b-8b0c-5d442a5e3506 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14552025-01-16T20:01:14.120ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1340e20f-2c0a-414e-bd48-99b3a410dce2 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14562025-01-16T20:01:14.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1340e20f-2c0a-414e-bd48-99b3a410dce2 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14572025-01-16T20:01:14.122ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2293 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1340e20f-2c0a-414e-bd48-99b3a410dce2 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14582025-01-16T20:01:15.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 935b4814-be16-478c-8588-fda3d4190898 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14592025-01-16T20:01:15.141ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 935b4814-be16-478c-8588-fda3d4190898 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14602025-01-16T20:01:15.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2693 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 935b4814-be16-478c-8588-fda3d4190898 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14612025-01-16T20:01:15.158ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 150740cc-8a42-45f0-8e3e-937f0142d782 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
14622025-01-16T20:01:15.158ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 150740cc-8a42-45f0-8e3e-937f0142d782 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
14632025-01-16T20:01:15.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3556 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 150740cc-8a42-45f0-8e3e-937f0142d782 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
14642025-01-16T20:01:16.162ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2c2207af-24b0-4d50-b05e-4d9755ff179f roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14652025-01-16T20:01:16.162ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2c2207af-24b0-4d50-b05e-4d9755ff179f resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14662025-01-16T20:01:16.164ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2136 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2c2207af-24b0-4d50-b05e-4d9755ff179f response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14672025-01-16T20:01:17.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8ea63163-f2b8-4082-a6e7-4bae230293ab roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14682025-01-16T20:01:17.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8ea63163-f2b8-4082-a6e7-4bae230293ab resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14692025-01-16T20:01:17.187ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2735 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8ea63163-f2b8-4082-a6e7-4bae230293ab response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14702025-01-16T20:01:18.206ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ff4f45ea-99d4-4642-910e-19ed6130ba74 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14712025-01-16T20:01:18.206ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ff4f45ea-99d4-4642-910e-19ed6130ba74 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14722025-01-16T20:01:18.208ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2315 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ff4f45ea-99d4-4642-910e-19ed6130ba74 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14732025-01-16T20:01:19.227ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 686244bd-a48e-48dc-b43b-e2ed36c977bc roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14742025-01-16T20:01:19.228ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 686244bd-a48e-48dc-b43b-e2ed36c977bc resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14752025-01-16T20:01:19.229ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2374 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 686244bd-a48e-48dc-b43b-e2ed36c977bc response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14762025-01-16T20:01:20.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ce153328-8e9b-43f7-8197-1c5a729f2772 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14772025-01-16T20:01:20.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ce153328-8e9b-43f7-8197-1c5a729f2772 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14782025-01-16T20:01:20.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2826 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ce153328-8e9b-43f7-8197-1c5a729f2772 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14792025-01-16T20:01:21.271ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f6e709c7-4c4b-4ce2-a022-6c8a675a272d roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14802025-01-16T20:01:21.271ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f6e709c7-4c4b-4ce2-a022-6c8a675a272d resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14812025-01-16T20:01:21.273ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2047 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f6e709c7-4c4b-4ce2-a022-6c8a675a272d response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14822025-01-16T20:01:22.292ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 72f1f06d-2b74-4965-af8c-dd6651134b4c roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14832025-01-16T20:01:22.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 72f1f06d-2b74-4965-af8c-dd6651134b4c resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14842025-01-16T20:01:22.295ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3028 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 72f1f06d-2b74-4965-af8c-dd6651134b4c response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
14852025-01-16T20:01:22.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
14862025-01-16T20:01:22.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
14872025-01-16T20:01:22.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14882025-01-16T20:01:22.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14892025-01-16T20:01:22.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14902025-01-16T20:01:22.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14912025-01-16T20:01:22.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14922025-01-16T20:01:22.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14932025-01-16T20:01:22.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14942025-01-16T20:01:22.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14952025-01-16T20:01:22.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14962025-01-16T20:01:22.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14972025-01-16T20:01:22.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14982025-01-16T20:01:22.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14992025-01-16T20:01:22.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15002025-01-16T20:01:22.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
15012025-01-16T20:01:22.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15022025-01-16T20:01:22.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15032025-01-16T20:01:22.323ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
15042025-01-16T20:01:22.323ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.91436ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
15052025-01-16T20:01:22.331ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.027402ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15062025-01-16T20:01:22.679ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 557 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = e73a33cf-096a-4055-b30a-f38fba52dd13 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
15072025-01-16T20:01:23.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = cee08f16-c2f9-4863-baa7-174e94d6c01e roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15082025-01-16T20:01:23.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = cee08f16-c2f9-4863-baa7-174e94d6c01e resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15092025-01-16T20:01:23.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2728 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = cee08f16-c2f9-4863-baa7-174e94d6c01e response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15102025-01-16T20:01:24.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 654a77cb-b491-47c7-90d8-564504cc2b12 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15112025-01-16T20:01:24.339ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 654a77cb-b491-47c7-90d8-564504cc2b12 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15122025-01-16T20:01:24.341ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2598 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 654a77cb-b491-47c7-90d8-564504cc2b12 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15132025-01-16T20:01:25.360ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ae079339-406b-455a-a764-c0750dd5e0b7 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15142025-01-16T20:01:25.360ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ae079339-406b-455a-a764-c0750dd5e0b7 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15152025-01-16T20:01:25.361ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1562 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ae079339-406b-455a-a764-c0750dd5e0b7 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15162025-01-16T20:01:26.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4f5d0f3b-3984-4430-8ca4-d5b837dbc7ed roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15172025-01-16T20:01:26.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4f5d0f3b-3984-4430-8ca4-d5b837dbc7ed resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15182025-01-16T20:01:26.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2582 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4f5d0f3b-3984-4430-8ca4-d5b837dbc7ed response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15192025-01-16T20:01:27.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f7c9c5c8-b8f2-451a-b8fb-45b8cf21da4c roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15202025-01-16T20:01:27.403ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f7c9c5c8-b8f2-451a-b8fb-45b8cf21da4c resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15212025-01-16T20:01:27.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2473 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f7c9c5c8-b8f2-451a-b8fb-45b8cf21da4c response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15222025-01-16T20:01:28.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5839caa3-48b2-4aab-ae7a-d2b77fb26ff4 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15232025-01-16T20:01:28.427ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5839caa3-48b2-4aab-ae7a-d2b77fb26ff4 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15242025-01-16T20:01:28.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2877 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5839caa3-48b2-4aab-ae7a-d2b77fb26ff4 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15252025-01-16T20:01:29.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b22225df-a4cf-49bb-b6d8-62ff2dc49875 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15262025-01-16T20:01:29.449ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b22225df-a4cf-49bb-b6d8-62ff2dc49875 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15272025-01-16T20:01:29.451ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2446 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b22225df-a4cf-49bb-b6d8-62ff2dc49875 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15282025-01-16T20:01:30.470ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7d0c4e8e-b03c-42c4-b790-79be957e7b39 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15292025-01-16T20:01:30.471ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7d0c4e8e-b03c-42c4-b790-79be957e7b39 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15302025-01-16T20:01:30.472ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2380 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7d0c4e8e-b03c-42c4-b790-79be957e7b39 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15312025-01-16T20:01:31.492ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 45bddd2c-2cd7-42e3-9e35-aed3c24ae505 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15322025-01-16T20:01:31.492ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 45bddd2c-2cd7-42e3-9e35-aed3c24ae505 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15332025-01-16T20:01:31.494ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2369 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 45bddd2c-2cd7-42e3-9e35-aed3c24ae505 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
15342025-01-16T20:01:32.299ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
15352025-01-16T20:01:32.299ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
15362025-01-16T20:01:32.300ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15372025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15382025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15392025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
15402025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
15412025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15422025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15432025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
15442025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15452025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
15462025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15472025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
15482025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
15492025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15502025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
15512025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
15522025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
15532025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15542025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15552025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15562025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
15572025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15582025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15592025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
15602025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15612025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
15622025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15632025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15642025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15652025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15662025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15672025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
15682025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15692025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15702025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15712025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15722025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15732025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15742025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15752025-01-16T20:01:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15762025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
15772025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
15782025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15792025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15802025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
15812025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15822025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
15832025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15842025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15852025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15862025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15872025-01-16T20:01:32.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15882025-01-16T20:01:32.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15892025-01-16T20:01:32.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15902025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15912025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15922025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15932025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15942025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15952025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15962025-01-16T20:01:32.304ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.877046ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
15972025-01-16T20:01:32.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15982025-01-16T20:01:32.305ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15992025-01-16T20:01:32.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16002025-01-16T20:01:32.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16012025-01-16T20:01:32.306ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16022025-01-16T20:01:32.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16032025-01-16T20:01:32.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16042025-01-16T20:01:32.307ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 539653083 } })
16052025-01-16T20:01:32.307ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1104
16062025-01-16T20:01:32.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16072025-01-16T20:01:32.308ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16082025-01-16T20:01:32.309ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 541281531 } })
16092025-01-16T20:01:32.309ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1104
16102025-01-16T20:01:32.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16112025-01-16T20:01:32.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16122025-01-16T20:01:32.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16132025-01-16T20:01:32.309ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16142025-01-16T20:01:32.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16152025-01-16T20:01:32.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16162025-01-16T20:01:32.310ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16172025-01-16T20:01:32.310ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16182025-01-16T20:01:32.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16192025-01-16T20:01:32.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16202025-01-16T20:01:32.311ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16212025-01-16T20:01:32.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16222025-01-16T20:01:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16232025-01-16T20:01:32.312ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16242025-01-16T20:01:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16252025-01-16T20:01:32.312ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.484492ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
16262025-01-16T20:01:32.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.906389ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
16272025-01-16T20:01:32.313ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16282025-01-16T20:01:32.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16292025-01-16T20:01:32.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16302025-01-16T20:01:32.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16312025-01-16T20:01:32.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.398963ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16322025-01-16T20:01:32.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16332025-01-16T20:01:32.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16342025-01-16T20:01:32.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16352025-01-16T20:01:32.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16362025-01-16T20:01:32.315ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 547920598 } })
16372025-01-16T20:01:32.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16382025-01-16T20:01:32.315ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1104 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
16392025-01-16T20:01:32.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16402025-01-16T20:01:32.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16412025-01-16T20:01:32.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16422025-01-16T20:01:32.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16432025-01-16T20:01:32.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16442025-01-16T20:01:32.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16452025-01-16T20:01:32.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16462025-01-16T20:01:32.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16472025-01-16T20:01:32.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16482025-01-16T20:01:32.320ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16492025-01-16T20:01:32.320ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 19.94688ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16502025-01-16T20:01:32.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16512025-01-16T20:01:32.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16522025-01-16T20:01:32.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16532025-01-16T20:01:32.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
16542025-01-16T20:01:32.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16552025-01-16T20:01:32.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16562025-01-16T20:01:32.325ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16572025-01-16T20:01:32.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16582025-01-16T20:01:32.327ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16592025-01-16T20:01:32.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16602025-01-16T20:01:32.328ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16612025-01-16T20:01:32.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16622025-01-16T20:01:32.330ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16632025-01-16T20:01:32.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16642025-01-16T20:01:32.331ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16652025-01-16T20:01:32.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16662025-01-16T20:01:32.334ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
16672025-01-16T20:01:32.334ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.77111ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
16682025-01-16T20:01:32.345ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16692025-01-16T20:01:32.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16702025-01-16T20:01:32.345ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16712025-01-16T20:01:32.345ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
16722025-01-16T20:01:32.345ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16732025-01-16T20:01:32.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16742025-01-16T20:01:32.346ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16752025-01-16T20:01:32.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16762025-01-16T20:01:32.348ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16772025-01-16T20:01:32.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16782025-01-16T20:01:32.351ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16792025-01-16T20:01:32.353ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 52.338048ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16802025-01-16T20:01:32.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87207105-a825-4eaa-ac6d-6aaa586b0fc5, lookup_type: ById(87207105-a825-4eaa-ac6d-6aaa586b0fc5) } result = Ok(())
16812025-01-16T20:01:32.353ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16822025-01-16T20:01:32.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16832025-01-16T20:01:32.354ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 54.862803ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16842025-01-16T20:01:32.355ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 55.565655ms iteration = 26 status = 0
16852025-01-16T20:01:32.355ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16862025-01-16T20:01:32.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16872025-01-16T20:01:32.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16882025-01-16T20:01:32.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 58.072649ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16892025-01-16T20:01:32.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16902025-01-16T20:01:32.359ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16912025-01-16T20:01:32.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16922025-01-16T20:01:32.361ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:370
16932025-01-16T20:01:32.361ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 61.112842ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16942025-01-16T20:01:32.391ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
16952025-01-16T20:01:32.391ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16962025-01-16T20:01:32.391ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "71e64611-f16d-4e0d-8534-558e519cd5b4", "content-length": "26", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
16972025-01-16T20:01:32.391ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16982025-01-16T20:01:32.392ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
16992025-01-16T20:01:32.392ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17002025-01-16T20:01:32.392ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7491f1f6-7baf-436a-8486-d6ef804773d1", "content-length": "26", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17012025-01-16T20:01:32.392ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17022025-01-16T20:01:32.392ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17032025-01-16T20:01:32.392ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
17042025-01-16T20:01:32.392ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17052025-01-16T20:01:32.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17062025-01-16T20:01:32.393ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17072025-01-16T20:01:32.394ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17082025-01-16T20:01:32.397ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1112 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f zone_address = fd00:1122:3344:101::2
17092025-01-16T20:01:32.397ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17102025-01-16T20:01:32.397ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec9c2bd1-87e9-4f65-84a7-d1196495af2a", "content-length": "26", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17112025-01-16T20:01:32.397ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1117 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17122025-01-16T20:01:32.399ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
17132025-01-16T20:01:32.422ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17142025-01-16T20:01:32.422ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
17152025-01-16T20:01:32.422ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 122.762994ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
17162025-01-16T20:01:32.431ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 131.61447ms iteration = 29 status = {"physical_disks_added":0}
17172025-01-16T20:01:32.514ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1e6fca28-0c1f-46e7-8421-866cbe2f405c roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17182025-01-16T20:01:32.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1e6fca28-0c1f-46e7-8421-866cbe2f405c resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17192025-01-16T20:01:32.516ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2078 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1e6fca28-0c1f-46e7-8421-866cbe2f405c response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17202025-01-16T20:01:32.530ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
17212025-01-16T20:01:32.530ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d624e6e-d5e6-4e68-9b5e-c56a85b8aed4", "content-length": "2", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17222025-01-16T20:01:32.531ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17232025-01-16T20:01:32.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17242025-01-16T20:01:32.536ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
17252025-01-16T20:01:32.536ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17262025-01-16T20:01:32.536ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4445070-e4a4-4e21-9055-8447fa04536c", "content-length": "372", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17272025-01-16T20:01:32.536ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17282025-01-16T20:01:32.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17292025-01-16T20:01:32.538ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 238.008196ms iteration = 26 status = null
17302025-01-16T20:01:32.541ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 241.398836ms iteration = 26 status = {}
17312025-01-16T20:01:32.541ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17322025-01-16T20:01:32.542ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb335c0e-3f2b-4585-a77c-7646def6311f", "content-length": "691", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17332025-01-16T20:01:32.542ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
17342025-01-16T20:01:32.542ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
17352025-01-16T20:01:32.542ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
17362025-01-16T20:01:32.543ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 243.966971ms iteration = 27 status = {}
17372025-01-16T20:01:32.584ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
17382025-01-16T20:01:32.584ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17392025-01-16T20:01:32.584ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c741cd01-7dac-47c9-8d9b-7d9a4cb4d2a8", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17402025-01-16T20:01:32.584ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17412025-01-16T20:01:32.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17422025-01-16T20:01:32.587ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17432025-01-16T20:01:32.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17442025-01-16T20:01:32.589ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17452025-01-16T20:01:32.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17462025-01-16T20:01:32.596ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17472025-01-16T20:01:32.596ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17482025-01-16T20:01:32.596ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "973a0ca1-adde-4876-b8d3-95c1a4f4c4aa", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17492025-01-16T20:01:32.598ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 298.960105ms iteration = 27 status = {}
17502025-01-16T20:01:32.603ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17512025-01-16T20:01:32.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17522025-01-16T20:01:32.606ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6ec20bf0-03fa-4e36-9de7-54dd5e9cfde1, rack_id: 26dc04a7-8537-47d8-965d-0931849f651f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(438946af-d2bd-4eab-8036-57fe8293b9f9) } rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
17532025-01-16T20:01:32.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17542025-01-16T20:01:32.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17552025-01-16T20:01:32.615ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
17562025-01-16T20:01:32.616ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8da5438f-5c18-4d41-9d9d-0aaa6a3112de", "content-length": "134", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17572025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
17582025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 438946af-d2bd-4eab-8036-57fe8293b9f9
17592025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17602025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17612025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17622025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17632025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17642025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17652025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17662025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17672025-01-16T20:01:32.616ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
17682025-01-16T20:01:32.616ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17692025-01-16T20:01:32.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17702025-01-16T20:01:32.619ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
17712025-01-16T20:01:32.619ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6c19e56-1262-44bf-8b80-7ead89a02243", "content-length": "2", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17722025-01-16T20:01:32.619ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
17732025-01-16T20:01:32.619ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "7453682e-e9df-41e5-9cc1-1d417f5bbff6", "content-length": "52", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17742025-01-16T20:01:32.619ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
17752025-01-16T20:01:32.619ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17762025-01-16T20:01:32.679ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 661 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = fa46b04d-2c49-4af3-a374-42edcd88774c response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
17772025-01-16T20:01:32.736ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5fac2a69-55fa-4505-b8b4-2285ca68ac53", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17782025-01-16T20:01:32.736ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch = Switch0
17792025-01-16T20:01:32.736ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17802025-01-16T20:01:32.737ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d3247f3-fa74-4b50-9322-9adbfe5d1c4b", "content-length": "682", "date": "Thu, 16 Jan 2025 20:01:32 GMT"} })
17812025-01-16T20:01:32.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17822025-01-16T20:01:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17832025-01-16T20:01:32.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17842025-01-16T20:01:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17852025-01-16T20:01:32.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17862025-01-16T20:01:32.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17872025-01-16T20:01:32.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17882025-01-16T20:01:32.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17892025-01-16T20:01:32.754ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 454.76527ms iteration = 26 status = {}
17902025-01-16T20:01:33.536ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ec8efdae-403d-40aa-a2dd-ca09afced1f0 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17912025-01-16T20:01:33.537ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ec8efdae-403d-40aa-a2dd-ca09afced1f0 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17922025-01-16T20:01:33.538ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1850 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ec8efdae-403d-40aa-a2dd-ca09afced1f0 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17932025-01-16T20:01:34.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 14035660-4a9a-4fdf-ba93-3e6bca9ad4b6 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17942025-01-16T20:01:34.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 14035660-4a9a-4fdf-ba93-3e6bca9ad4b6 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17952025-01-16T20:01:34.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2310 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 14035660-4a9a-4fdf-ba93-3e6bca9ad4b6 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17962025-01-16T20:01:35.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0f03ad9d-cdc4-474c-9c39-d2ff56df2d89 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17972025-01-16T20:01:35.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0f03ad9d-cdc4-474c-9c39-d2ff56df2d89 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17982025-01-16T20:01:35.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2515 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0f03ad9d-cdc4-474c-9c39-d2ff56df2d89 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
17992025-01-16T20:01:36.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2a0b827c-71bc-4ff6-abc8-725862af8ca5 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18002025-01-16T20:01:36.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2a0b827c-71bc-4ff6-abc8-725862af8ca5 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18012025-01-16T20:01:36.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2191 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 2a0b827c-71bc-4ff6-abc8-725862af8ca5 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18022025-01-16T20:01:37.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 75116ba8-d809-4f79-9ec2-19167c1e85b5 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18032025-01-16T20:01:37.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 75116ba8-d809-4f79-9ec2-19167c1e85b5 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18042025-01-16T20:01:37.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2160 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 75116ba8-d809-4f79-9ec2-19167c1e85b5 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18052025-01-16T20:01:38.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7b0f371f-0148-4023-8809-09fd15cfe782 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18062025-01-16T20:01:38.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7b0f371f-0148-4023-8809-09fd15cfe782 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18072025-01-16T20:01:38.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2087 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 7b0f371f-0148-4023-8809-09fd15cfe782 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18082025-01-16T20:01:39.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0d2c00a8-98d2-4a37-bb38-f33923de9b16 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18092025-01-16T20:01:39.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0d2c00a8-98d2-4a37-bb38-f33923de9b16 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18102025-01-16T20:01:39.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2541 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0d2c00a8-98d2-4a37-bb38-f33923de9b16 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18112025-01-16T20:01:40.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b5e4bf6b-8e54-4c98-85b1-b99fc3f85a8f roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18122025-01-16T20:01:40.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b5e4bf6b-8e54-4c98-85b1-b99fc3f85a8f resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18132025-01-16T20:01:40.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2280 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = b5e4bf6b-8e54-4c98-85b1-b99fc3f85a8f response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18142025-01-16T20:01:41.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d3e03a3c-54c5-4d19-a3b4-a3ccfcbd6a21 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18152025-01-16T20:01:41.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d3e03a3c-54c5-4d19-a3b4-a3ccfcbd6a21 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18162025-01-16T20:01:41.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2423 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d3e03a3c-54c5-4d19-a3b4-a3ccfcbd6a21 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18172025-01-16T20:01:42.300ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
18182025-01-16T20:01:42.300ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
18192025-01-16T20:01:42.300ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18202025-01-16T20:01:42.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18212025-01-16T20:01:42.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18222025-01-16T20:01:42.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18232025-01-16T20:01:42.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242025-01-16T20:01:42.303ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18252025-01-16T20:01:42.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18262025-01-16T20:01:42.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18272025-01-16T20:01:42.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18282025-01-16T20:01:42.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.979267ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18292025-01-16T20:01:42.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18302025-01-16T20:01:42.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18312025-01-16T20:01:42.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18322025-01-16T20:01:42.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18332025-01-16T20:01:42.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
18342025-01-16T20:01:42.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18352025-01-16T20:01:42.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18362025-01-16T20:01:42.320ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
18372025-01-16T20:01:42.320ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.819308ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
18382025-01-16T20:01:42.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 4cad5a19-d0c1-4338-9218-eedf6cf5de11 roles = RoleSet { roles: {} } uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
18392025-01-16T20:01:42.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 4cad5a19-d0c1-4338-9218-eedf6cf5de11 resource = Database result = Ok(()) uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
18402025-01-16T20:01:42.674ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3194 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = 4cad5a19-d0c1-4338-9218-eedf6cf5de11 response_code = 200 uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?sort_by=id_ascending
18412025-01-16T20:01:42.674ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = fa98dfe9-b31d-4468-beaa-21b95c1f80bd roles = RoleSet { roles: {} } uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
18422025-01-16T20:01:42.675ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = fa98dfe9-b31d-4468-beaa-21b95c1f80bd resource = Database result = Ok(()) uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
18432025-01-16T20:01:42.677ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2390 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44602 req_id = fa98dfe9-b31d-4468-beaa-21b95c1f80bd response_code = 200 uri = /metrics/collectors/d048ac10-211f-4354-a749-75b8d62cfaed/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlZWFjMTliZS0yMzc3LTQyMDItYmQzYS0xNjEyMjBiMjRkNDAifX0%3D
18442025-01-16T20:01:42.678ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 545 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = d861427a-1798-403b-a28b-b9fb0e0b996b response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
18452025-01-16T20:01:42.729ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c16d2264-ee15-475b-bb18-2ae074bd6af3 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18462025-01-16T20:01:42.730ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c16d2264-ee15-475b-bb18-2ae074bd6af3 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18472025-01-16T20:01:42.731ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1870 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c16d2264-ee15-475b-bb18-2ae074bd6af3 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18482025-01-16T20:01:43.751ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8b88e9ec-71df-4764-9bc8-af140a64790f roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18492025-01-16T20:01:43.751ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8b88e9ec-71df-4764-9bc8-af140a64790f resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18502025-01-16T20:01:43.753ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2164 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 8b88e9ec-71df-4764-9bc8-af140a64790f response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18512025-01-16T20:01:44.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 00bd4335-aa07-4742-a01e-a46803b96ae4 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18522025-01-16T20:01:44.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 00bd4335-aa07-4742-a01e-a46803b96ae4 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18532025-01-16T20:01:44.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2414 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 00bd4335-aa07-4742-a01e-a46803b96ae4 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18542025-01-16T20:01:45.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 26cc2ced-5616-4020-a838-7ef9f9024b01 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
18552025-01-16T20:01:45.193ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 26cc2ced-5616-4020-a838-7ef9f9024b01 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
18562025-01-16T20:01:45.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3682 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:58450 req_id = 26cc2ced-5616-4020-a838-7ef9f9024b01 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
18572025-01-16T20:01:45.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 17d968b0-0719-4dfc-b20e-19dab7259416 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18582025-01-16T20:01:45.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 17d968b0-0719-4dfc-b20e-19dab7259416 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18592025-01-16T20:01:45.797ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2423 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 17d968b0-0719-4dfc-b20e-19dab7259416 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18602025-01-16T20:01:46.815ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 51b2e21c-7236-4cae-be33-deb0400848a3 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18612025-01-16T20:01:46.816ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 51b2e21c-7236-4cae-be33-deb0400848a3 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18622025-01-16T20:01:46.818ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2605 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 51b2e21c-7236-4cae-be33-deb0400848a3 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18632025-01-16T20:01:47.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c8bf2879-b81e-4da5-9224-932c82a89ba8 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18642025-01-16T20:01:47.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c8bf2879-b81e-4da5-9224-932c82a89ba8 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18652025-01-16T20:01:47.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2428 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = c8bf2879-b81e-4da5-9224-932c82a89ba8 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18662025-01-16T20:01:48.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5262b7f2-7e34-46ec-b9f1-3cd02df369b0 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18672025-01-16T20:01:48.859ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5262b7f2-7e34-46ec-b9f1-3cd02df369b0 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18682025-01-16T20:01:48.861ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2507 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 5262b7f2-7e34-46ec-b9f1-3cd02df369b0 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18692025-01-16T20:01:49.879ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d4d980a5-ab25-4ce0-8a77-1890c2c03777 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18702025-01-16T20:01:49.880ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d4d980a5-ab25-4ce0-8a77-1890c2c03777 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18712025-01-16T20:01:49.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2456 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d4d980a5-ab25-4ce0-8a77-1890c2c03777 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18722025-01-16T20:01:50.901ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ca1ff1fe-4ec4-42f6-9a43-0124eaccc5e5 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18732025-01-16T20:01:50.901ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ca1ff1fe-4ec4-42f6-9a43-0124eaccc5e5 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18742025-01-16T20:01:50.903ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2507 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = ca1ff1fe-4ec4-42f6-9a43-0124eaccc5e5 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18752025-01-16T20:01:51.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d6993192-c593-48dd-9979-0d4b53b20d4a roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18762025-01-16T20:01:51.923ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d6993192-c593-48dd-9979-0d4b53b20d4a resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18772025-01-16T20:01:51.925ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2420 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = d6993192-c593-48dd-9979-0d4b53b20d4a response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
18782025-01-16T20:01:52.299ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
18792025-01-16T20:01:52.299ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
18802025-01-16T20:01:52.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18812025-01-16T20:01:52.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18822025-01-16T20:01:52.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18832025-01-16T20:01:52.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18842025-01-16T20:01:52.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18852025-01-16T20:01:52.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18862025-01-16T20:01:52.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18872025-01-16T20:01:52.303ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.745175ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18882025-01-16T20:01:52.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18892025-01-16T20:01:52.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18902025-01-16T20:01:52.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18912025-01-16T20:01:52.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18922025-01-16T20:01:52.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18932025-01-16T20:01:52.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18942025-01-16T20:01:52.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
18952025-01-16T20:01:52.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18962025-01-16T20:01:52.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18972025-01-16T20:01:52.322ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
18982025-01-16T20:01:52.322ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.428614ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
18992025-01-16T20:01:52.679ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 921 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 28ed8f06-f236-4467-ad8e-dd4e9a6121c9 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
19002025-01-16T20:01:52.944ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4e748548-f1a8-429b-9cb9-842357bcef31 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19012025-01-16T20:01:52.945ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4e748548-f1a8-429b-9cb9-842357bcef31 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19022025-01-16T20:01:52.947ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2476 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 4e748548-f1a8-429b-9cb9-842357bcef31 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19032025-01-16T20:01:53.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f3907241-bb99-48c4-81bc-81d46e90c4e6 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19042025-01-16T20:01:53.967ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f3907241-bb99-48c4-81bc-81d46e90c4e6 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19052025-01-16T20:01:53.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1997 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = f3907241-bb99-48c4-81bc-81d46e90c4e6 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19062025-01-16T20:01:54.989ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1579ccf1-53ba-4f0f-9a25-0c453fb8919c roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19072025-01-16T20:01:54.989ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1579ccf1-53ba-4f0f-9a25-0c453fb8919c resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19082025-01-16T20:01:54.991ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2758 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 1579ccf1-53ba-4f0f-9a25-0c453fb8919c response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19092025-01-16T20:01:56.009ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a5c58ee0-752a-4b66-8137-5510b7bdfe38 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19102025-01-16T20:01:56.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a5c58ee0-752a-4b66-8137-5510b7bdfe38 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19112025-01-16T20:01:56.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2491 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = a5c58ee0-752a-4b66-8137-5510b7bdfe38 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19122025-01-16T20:01:57.030ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0bfb2e2b-a32d-4bee-9960-63b8ce4c2169 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19132025-01-16T20:01:57.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0bfb2e2b-a32d-4bee-9960-63b8ce4c2169 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19142025-01-16T20:01:57.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2580 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0bfb2e2b-a32d-4bee-9960-63b8ce4c2169 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19152025-01-16T20:01:58.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0586b3cf-c322-4753-9d54-bb404ed6f09f roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19162025-01-16T20:01:58.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0586b3cf-c322-4753-9d54-bb404ed6f09f resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19172025-01-16T20:01:58.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2441 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 0586b3cf-c322-4753-9d54-bb404ed6f09f response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19182025-01-16T20:01:59.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 448b5845-dbfe-4064-96c2-5668bf2148b5 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19192025-01-16T20:01:59.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 448b5845-dbfe-4064-96c2-5668bf2148b5 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19202025-01-16T20:01:59.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1859 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 448b5845-dbfe-4064-96c2-5668bf2148b5 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19212025-01-16T20:02:00.094ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 79748cbe-d4de-47d7-af24-b22b5162829a roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19222025-01-16T20:02:00.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 79748cbe-d4de-47d7-af24-b22b5162829a resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19232025-01-16T20:02:00.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3327 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 79748cbe-d4de-47d7-af24-b22b5162829a response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19242025-01-16T20:02:01.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 160e90a2-5261-4de4-854e-28e41e138e71 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19252025-01-16T20:02:01.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 160e90a2-5261-4de4-854e-28e41e138e71 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19262025-01-16T20:02:01.119ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1948 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 160e90a2-5261-4de4-854e-28e41e138e71 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19272025-01-16T20:02:02.138ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e674ed1a-a88e-4ec3-8e73-5f9fb243a859 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19282025-01-16T20:02:02.139ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e674ed1a-a88e-4ec3-8e73-5f9fb243a859 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19292025-01-16T20:02:02.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2839 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = e674ed1a-a88e-4ec3-8e73-5f9fb243a859 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
19302025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
19312025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
19322025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
19332025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19342025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
19352025-01-16T20:02:02.300ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-01-16 19:52:02.300757211 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
19362025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
19372025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
19382025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19392025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
19402025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
19412025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
19422025-01-16T20:02:02.300ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8
19432025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
19442025-01-16T20:02:02.300ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
19452025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.32\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8"}
19462025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
19472025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19482025-01-16T20:02:02.300ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
19492025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19502025-01-16T20:02:02.301ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19512025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
19522025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
19532025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19542025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19552025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
19562025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19572025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
19582025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19592025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19602025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
19612025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19622025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
19632025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
19642025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19652025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
19662025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19672025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
19682025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19692025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19702025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
19712025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19722025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
19732025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19742025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
19752025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19762025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
19772025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
19782025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
19792025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
19802025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
19812025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
19822025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
19832025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19842025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19852025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
19862025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
19872025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19882025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19892025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
19902025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
19912025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19922025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
19932025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19942025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
19952025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19962025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19972025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19982025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19992025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20002025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20012025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
20022025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
20032025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20042025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20052025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20062025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20072025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
20082025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20092025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20102025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20112025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20122025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
20132025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20142025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20152025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20162025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20172025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20182025-01-16T20:02:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20192025-01-16T20:02:02.301ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("67baed66-640b-413d-8436-22e580c71fff.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b6525081-e08a-4759-918e-50f60ff0f6f1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b6525081-e08a-4759-918e-50f60ff0f6f1.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 533695018 } })
20202025-01-16T20:02:02.301ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1673b82a-d301-40a0-97f5-01c1919bcccc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a78ec168-75b0-4c15-b272-0f34e450b715.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fc5e4155-1b26-42b3-8db4-a4cf6dd61fe9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fc5e4155-1b26-42b3-8db4-a4cf6dd61fe9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 533724279 } })
20212025-01-16T20:02:02.302ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
20222025-01-16T20:02:02.302ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.405664ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
20232025-01-16T20:02:02.302ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
20242025-01-16T20:02:02.302ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.483337ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
20252025-01-16T20:02:02.304ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
20262025-01-16T20:02:02.304ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 536600359 } })
20272025-01-16T20:02:02.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20282025-01-16T20:02:02.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20292025-01-16T20:02:02.304ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1104
20302025-01-16T20:02:02.305ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.940879ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20312025-01-16T20:02:02.305ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 537060637 } })
20322025-01-16T20:02:02.305ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1104
20332025-01-16T20:02:02.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20342025-01-16T20:02:02.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20352025-01-16T20:02:02.306ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20362025-01-16T20:02:02.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20372025-01-16T20:02:02.306ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20382025-01-16T20:02:02.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.26437ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
20392025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
20402025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20412025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20422025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20432025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20442025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20452025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20462025-01-16T20:02:02.308ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-01-16 19:52:02.300757211 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
20472025-01-16T20:02:02.308ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.838847ms iteration = 14 status = {"expiration":"2025-01-16T19:52:02.300757211Z","pruned":[]}
20482025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 9.271413ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
20492025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20502025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20512025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20522025-01-16T20:02:02.310ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20532025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20542025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20552025-01-16T20:02:02.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20562025-01-16T20:02:02.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20572025-01-16T20:02:02.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20582025-01-16T20:02:02.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20592025-01-16T20:02:02.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20602025-01-16T20:02:02.313ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.394388ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
20612025-01-16T20:02:02.313ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20622025-01-16T20:02:02.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20632025-01-16T20:02:02.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20642025-01-16T20:02:02.314ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20652025-01-16T20:02:02.314ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20662025-01-16T20:02:02.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20672025-01-16T20:02:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20682025-01-16T20:02:02.316ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20692025-01-16T20:02:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20702025-01-16T20:02:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20712025-01-16T20:02:02.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20722025-01-16T20:02:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20732025-01-16T20:02:02.317ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20742025-01-16T20:02:02.317ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20752025-01-16T20:02:02.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20762025-01-16T20:02:02.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20772025-01-16T20:02:02.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20782025-01-16T20:02:02.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20792025-01-16T20:02:02.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20802025-01-16T20:02:02.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20812025-01-16T20:02:02.323ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20822025-01-16T20:02:02.325ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 23.070934ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20832025-01-16T20:02:02.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20842025-01-16T20:02:02.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20852025-01-16T20:02:02.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20862025-01-16T20:02:02.328ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20872025-01-16T20:02:02.328ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20882025-01-16T20:02:02.328ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20892025-01-16T20:02:02.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20902025-01-16T20:02:02.329ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20912025-01-16T20:02:02.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20922025-01-16T20:02:02.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20932025-01-16T20:02:02.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20942025-01-16T20:02:02.331ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20952025-01-16T20:02:02.331ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20962025-01-16T20:02:02.331ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 30.624757ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20972025-01-16T20:02:02.332ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20982025-01-16T20:02:02.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20992025-01-16T20:02:02.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21002025-01-16T20:02:02.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21012025-01-16T20:02:02.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
21022025-01-16T20:02:02.335ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21032025-01-16T20:02:02.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21042025-01-16T20:02:02.339ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21052025-01-16T20:02:02.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21062025-01-16T20:02:02.341ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21072025-01-16T20:02:02.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21082025-01-16T20:02:02.343ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21092025-01-16T20:02:02.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21102025-01-16T20:02:02.348ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
21112025-01-16T20:02:02.348ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
21122025-01-16T20:02:02.348ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 47.214578ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
21132025-01-16T20:02:02.352ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21142025-01-16T20:02:02.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21152025-01-16T20:02:02.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21162025-01-16T20:02:02.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21172025-01-16T20:02:02.356ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.263819ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21182025-01-16T20:02:02.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21192025-01-16T20:02:02.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21202025-01-16T20:02:02.358ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21212025-01-16T20:02:02.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21222025-01-16T20:02:02.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 57.825444ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21232025-01-16T20:02:02.359ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21242025-01-16T20:02:02.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21252025-01-16T20:02:02.362ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21262025-01-16T20:02:02.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21272025-01-16T20:02:02.363ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21282025-01-16T20:02:02.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21292025-01-16T20:02:02.366ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21302025-01-16T20:02:02.366ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
21312025-01-16T20:02:02.366ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 65.178704ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21322025-01-16T20:02:02.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21332025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87207105-a825-4eaa-ac6d-6aaa586b0fc5, lookup_type: ById(87207105-a825-4eaa-ac6d-6aaa586b0fc5) } result = Ok(())
21342025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21352025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21362025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21372025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21382025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21392025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21402025-01-16T20:02:02.367ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21412025-01-16T20:02:02.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 39de0ab7-8c61-46aa-88f2-d42db34dffb8, lookup_type: ById(39de0ab7-8c61-46aa-88f2-d42db34dffb8) } result = Ok(())
21422025-01-16T20:02:02.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21432025-01-16T20:02:02.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21442025-01-16T20:02:02.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21452025-01-16T20:02:02.368ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21462025-01-16T20:02:02.368ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21472025-01-16T20:02:02.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21482025-01-16T20:02:02.369ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 68.450271ms iteration = 27 status = 0
21492025-01-16T20:02:02.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21502025-01-16T20:02:02.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21512025-01-16T20:02:02.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
21522025-01-16T20:02:02.373ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21532025-01-16T20:02:02.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21542025-01-16T20:02:02.375ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21552025-01-16T20:02:02.376ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21562025-01-16T20:02:02.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21572025-01-16T20:02:02.378ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-87207105-a825-4eaa-ac6d-6aaa586b0fc5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 610183070 } })
21582025-01-16T20:02:02.378ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21592025-01-16T20:02:02.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21602025-01-16T20:02:02.378ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21612025-01-16T20:02:02.378ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1104 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
21622025-01-16T20:02:02.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21632025-01-16T20:02:02.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
21642025-01-16T20:02:02.380ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21652025-01-16T20:02:02.380ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21662025-01-16T20:02:02.380ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21672025-01-16T20:02:02.380ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 79.079697ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21682025-01-16T20:02:02.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21692025-01-16T20:02:02.381ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:370
21702025-01-16T20:02:02.381ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 80.759437ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
21712025-01-16T20:02:02.386ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21722025-01-16T20:02:02.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21732025-01-16T20:02:02.386ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 original_time_created = 2025-01-16 19:48:51.140214 UTC target_id = 39de0ab7-8c61-46aa-88f2-d42db34dffb8 time_created = 2025-01-16 19:48:51.140214 UTC
21742025-01-16T20:02:02.386ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 85.936638ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"39de0ab7-8c61-46aa-88f2-d42db34dffb8","time_created":"2025-01-16T19:48:51.140214Z"}
21752025-01-16T20:02:02.393ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 92.977111ms iteration = 30 status = {"physical_disks_added":0}
21762025-01-16T20:02:02.398ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-01-16T19:48:57.779461Z, time_modified: 2025-01-16T19:48:57.779461Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 197fa9bd-2b54-417a-8249-a5354e078485, db_silo: Silo { identity: SiloIdentity { id: 197fa9bd-2b54-417a-8249-a5354e078485, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-16T19:49:10.527303Z, time_modified: 2025-01-16T19:49:10.527303Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 197fa9bd-2b54-417a-8249-a5354e078485, db_silo: Silo { identity: SiloIdentity { id: 197fa9bd-2b54-417a-8249-a5354e078485, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-01-16T19:49:10.527303Z, time_modified: 2025-01-16T19:49:10.527303Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538" }] }) }
21772025-01-16T20:02:02.398ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 96.735007ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"197fa9bd-2b54-417a-8249-a5354e078485","tls_certs":["b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538"]}},"default_endpoint":{"silo_id":"197fa9bd-2b54-417a-8249-a5354e078485","tls_certs":["b24122d5f92c7bd84d282bd8841cee73a1e4d9ad23f30c8a432049fc1b2c4538"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
21782025-01-16T20:02:02.411ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
21792025-01-16T20:02:02.411ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 481516d2-3f4d-4dc5-9036-8c14a1fa4fd7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
21802025-01-16T20:02:02.411ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21812025-01-16T20:02:02.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21822025-01-16T20:02:02.412ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21832025-01-16T20:02:02.412ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21842025-01-16T20:02:02.412ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21852025-01-16T20:02:02.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21862025-01-16T20:02:02.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21872025-01-16T20:02:02.413ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21882025-01-16T20:02:02.414ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21892025-01-16T20:02:02.416ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.234692ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
21902025-01-16T20:02:02.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21912025-01-16T20:02:02.417ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21922025-01-16T20:02:02.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21932025-01-16T20:02:02.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21942025-01-16T20:02:02.417ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21952025-01-16T20:02:02.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21962025-01-16T20:02:02.422ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 481516d2-3f4d-4dc5-9036-8c14a1fa4fd7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
21972025-01-16T20:02:02.422ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
21982025-01-16T20:02:02.422ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-01-16 19:49:07.686045 UTC dns_group = internal generation = 1
21992025-01-16T20:02:02.422ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 121.601684ms iteration = 14 status = {"generation":1}
22002025-01-16T20:02:02.430ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22012025-01-16T20:02:02.438ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
22022025-01-16T20:02:02.438ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 138.018802ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
22032025-01-16T20:02:02.448ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
22042025-01-16T20:02:02.448ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22052025-01-16T20:02:02.448ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7888caa5-84a1-4f58-a197-fcba921a7cb2", "content-length": "26", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22062025-01-16T20:02:02.448ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22072025-01-16T20:02:02.448ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
22082025-01-16T20:02:02.449ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22092025-01-16T20:02:02.449ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = f05be43b-5d84-4b95-99be-d6d0a1e3a923 dns_zone_name = oxide.test found_so_far = 0 version = 2
22102025-01-16T20:02:02.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22112025-01-16T20:02:02.449ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22122025-01-16T20:02:02.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22132025-01-16T20:02:02.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22142025-01-16T20:02:02.454ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1112 zone_address = fd00:1122:3344:101::2
22152025-01-16T20:02:02.454ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22162025-01-16T20:02:02.454ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3570960-b886-4c27-b0ca-772455ba35f5", "content-length": "26", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22172025-01-16T20:02:02.454ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1117 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22182025-01-16T20:02:02.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22192025-01-16T20:02:02.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22202025-01-16T20:02:02.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22212025-01-16T20:02:02.456ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22222025-01-16T20:02:02.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22232025-01-16T20:02:02.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22242025-01-16T20:02:02.588ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22252025-01-16T20:02:02.588ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b3e2d074-ead3-4c55-b62a-9711e827c92a", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22262025-01-16T20:02:02.588ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22272025-01-16T20:02:02.590ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c6a2ebd3-a6a7-4b9e-acfb-afa5efcb9ec0", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22282025-01-16T20:02:02.590ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22292025-01-16T20:02:02.592ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22302025-01-16T20:02:02.592ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 291.441564ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22312025-01-16T20:02:02.622ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22322025-01-16T20:02:02.623ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e10f04e-f1bb-44b8-92f1-eb20efd6d767", "content-length": "2", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22332025-01-16T20:02:02.625ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22342025-01-16T20:02:02.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22352025-01-16T20:02:02.639ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1112 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f zone_address = fd00:1122:3344:101::2
22362025-01-16T20:02:02.639ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22372025-01-16T20:02:02.639ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5686b94d-64c0-4dc8-ba8c-48ed36562f96", "content-length": "26", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22382025-01-16T20:02:02.639ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1117 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22392025-01-16T20:02:02.642ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1149 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
22402025-01-16T20:02:02.679ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1035 local_addr = [fd00:1122:3344:101::a]:42645 method = GET remote_addr = [fd00:1122:3344:101::d]:57004 req_id = 064513b4-e01b-47a3-8354-1d1f3ee4e962 response_code = 200 uri = /4bd2ffa8-9f85-44e5-a021-54e6cdd59be5
22412025-01-16T20:02:02.693ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
22422025-01-16T20:02:02.693ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22432025-01-16T20:02:02.694ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "39a55714-4521-4089-92c4-ffb3afb7e3a9", "content-length": "372", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22442025-01-16T20:02:02.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22452025-01-16T20:02:02.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22462025-01-16T20:02:02.698ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = f05be43b-5d84-4b95-99be-d6d0a1e3a923 dns_zone_name = oxide.test found_so_far = 1 version = 2
22472025-01-16T20:02:02.698ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22482025-01-16T20:02:02.698ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-01-16 19:49:10.849798 UTC dns_group = external generation = 2
22492025-01-16T20:02:02.698ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 397.279691ms iteration = 14 status = {"generation":2}
22502025-01-16T20:02:02.709ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22512025-01-16T20:02:02.711ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 410.281599ms iteration = 27 status = null
22522025-01-16T20:02:02.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22532025-01-16T20:02:02.712ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22542025-01-16T20:02:02.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22552025-01-16T20:02:02.714ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22562025-01-16T20:02:02.716ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22572025-01-16T20:02:02.716ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22582025-01-16T20:02:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22592025-01-16T20:02:02.732ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22602025-01-16T20:02:02.750ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 450.021427ms iteration = 27 status = {}
22612025-01-16T20:02:02.757ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
22622025-01-16T20:02:02.757ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22632025-01-16T20:02:02.757ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "aad5167a-2763-49ab-8cb5-1a3931a7f453", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22642025-01-16T20:02:02.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22652025-01-16T20:02:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22662025-01-16T20:02:02.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22672025-01-16T20:02:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22682025-01-16T20:02:02.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22692025-01-16T20:02:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22702025-01-16T20:02:02.766ZINFOnexus (BackgroundTasks): VPC route manager sled 87207105-a825-4eaa-ac6d-6aaa586b0fc5 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22712025-01-16T20:02:02.766ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22722025-01-16T20:02:02.766ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f09fdac5-21df-4b54-a1df-4b8f95653ea8", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22732025-01-16T20:02:02.768ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 467.88972ms iteration = 28 status = {}
22742025-01-16T20:02:02.837ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22752025-01-16T20:02:02.838ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "38433aff-80f7-4472-871d-57b630cc51ed", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22762025-01-16T20:02:02.838ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22772025-01-16T20:02:02.839ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "216ed738-6748-4d56-a012-1b2fe6b2e3ba", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22782025-01-16T20:02:02.839ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22792025-01-16T20:02:02.842ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "552750a2-fb1f-443d-a05e-a64dcfcd785f", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22802025-01-16T20:02:02.842ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22812025-01-16T20:02:02.844ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22822025-01-16T20:02:02.844ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 543.592558ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
22832025-01-16T20:02:02.931ZDEBGnexus (BackgroundTasks): client request SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
22842025-01-16T20:02:02.932ZDEBGnexus (BackgroundTasks): client response SledAgent = 87207105-a825-4eaa-ac6d-6aaa586b0fc5 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8985ddc1-0afb-4e67-9f01-fa21d6050d7f", "content-length": "691", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22852025-01-16T20:02:02.932ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
22862025-01-16T20:02:02.932ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
22872025-01-16T20:02:02.932ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
22882025-01-16T20:02:02.934ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 633.417164ms iteration = 28 status = {}
22892025-01-16T20:02:02.963ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22902025-01-16T20:02:02.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22912025-01-16T20:02:02.966ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 6ec20bf0-03fa-4e36-9de7-54dd5e9cfde1, rack_id: 26dc04a7-8537-47d8-965d-0931849f651f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(438946af-d2bd-4eab-8036-57fe8293b9f9) } rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
22922025-01-16T20:02:02.966ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22932025-01-16T20:02:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22942025-01-16T20:02:02.974ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
22952025-01-16T20:02:02.974ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdf92c50-f1ee-4646-8691-96532bb61c6f", "content-length": "134", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
22962025-01-16T20:02:02.974ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
22972025-01-16T20:02:02.974ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 438946af-d2bd-4eab-8036-57fe8293b9f9
22982025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
22992025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23002025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23012025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23022025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23032025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23042025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23052025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23062025-01-16T20:02:02.975ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f
23072025-01-16T20:02:02.975ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23082025-01-16T20:02:02.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23092025-01-16T20:02:02.978ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23102025-01-16T20:02:02.978ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0248bb06-d1cd-4efb-81b7-430a6e1a0be9", "content-length": "2", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
23112025-01-16T20:02:02.978ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23122025-01-16T20:02:02.978ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2fbae3f-25f8-4f87-9276-d4577e79d452", "content-length": "52", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
23132025-01-16T20:02:02.978ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch_location = Switch0
23142025-01-16T20:02:02.978ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23152025-01-16T20:02:03.088ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6d4c103e-c8ba-458f-89cf-3a6a88ffccb8", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
23162025-01-16T20:02:03.088ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 26dc04a7-8537-47d8-965d-0931849f651f switch = Switch0
23172025-01-16T20:02:03.088ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 26dc04a7-8537-47d8-965d-0931849f651f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23182025-01-16T20:02:03.088ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 26dc04a7-8537-47d8-965d-0931849f651f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "066922f0-94b7-47c3-83f3-d36d1257c337", "content-length": "682", "date": "Thu, 16 Jan 2025 20:02:02 GMT"} })
23192025-01-16T20:02:03.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23202025-01-16T20:02:03.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23212025-01-16T20:02:03.093ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23222025-01-16T20:02:03.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23232025-01-16T20:02:03.097ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23242025-01-16T20:02:03.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23252025-01-16T20:02:03.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23262025-01-16T20:02:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23272025-01-16T20:02:03.106ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 804.764861ms iteration = 27 status = {}
23282025-01-16T20:02:03.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3dc9c111-3ae3-4674-b976-796357579857 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
23292025-01-16T20:02:03.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3dc9c111-3ae3-4674-b976-796357579857 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
23302025-01-16T20:02:03.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2229 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 3dc9c111-3ae3-4674-b976-796357579857 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
23312025-01-16T20:02:04.273ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 148fc5f5-f9d0-4bbb-90c2-66db1445fe59 roles = RoleSet { roles: {} } uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
23322025-01-16T20:02:04.274ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 148fc5f5-f9d0-4bbb-90c2-66db1445fe59 resource = Database result = Ok(()) uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5
23332025-01-16T20:02:04.276ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2595 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:45148 req_id = 148fc5f5-f9d0-4bbb-90c2-66db1445fe59 response_code = 200 uri = /probes/87207105-a825-4eaa-ac6d-6aaa586b0fc5