Max level shown:
12025-07-15T21:15:03.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b92f19f1-a319-436c-8809-82e7a05fe34e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
22025-07-15T21:15:03.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b92f19f1-a319-436c-8809-82e7a05fe34e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32025-07-15T21:15:03.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2600 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b92f19f1-a319-436c-8809-82e7a05fe34e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
42025-07-15T21:15:04.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f206a9b6-26df-4891-b962-2d7231b85635 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
52025-07-15T21:15:04.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f206a9b6-26df-4891-b962-2d7231b85635 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
62025-07-15T21:15:04.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2171 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f206a9b6-26df-4891-b962-2d7231b85635 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
72025-07-15T21:15:05.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7ea024f7-d436-44b0-99e5-b4475771ccf4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
82025-07-15T21:15:05.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7ea024f7-d436-44b0-99e5-b4475771ccf4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
92025-07-15T21:15:05.216ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2213 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7ea024f7-d436-44b0-99e5-b4475771ccf4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
102025-07-15T21:15:06.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 26bc0cfb-fc2a-4e4b-b480-7f252d3eedb0 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
112025-07-15T21:15:06.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 26bc0cfb-fc2a-4e4b-b480-7f252d3eedb0 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
122025-07-15T21:15:06.232ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 26bc0cfb-fc2a-4e4b-b480-7f252d3eedb0 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
132025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
142025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
152025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
162025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 107 reason = Timeout
182025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
192025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
202025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
212025-07-15T21:15:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
222025-07-15T21:15:06.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232025-07-15T21:15:06.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
242025-07-15T21:15:06.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
252025-07-15T21:15:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
262025-07-15T21:15:06.890ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.387743ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
272025-07-15T21:15:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
282025-07-15T21:15:06.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
292025-07-15T21:15:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
302025-07-15T21:15:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
312025-07-15T21:15:06.893ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
322025-07-15T21:15:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
332025-07-15T21:15:06.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
342025-07-15T21:15:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
352025-07-15T21:15:06.894ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
362025-07-15T21:15:06.894ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.386817ms iteration = 107 status = {"chicken_switches_updated":false}
372025-07-15T21:15:06.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
382025-07-15T21:15:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
392025-07-15T21:15:06.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
402025-07-15T21:15:06.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
412025-07-15T21:15:06.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
422025-07-15T21:15:06.909ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.533102ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
432025-07-15T21:15:07.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9d3c4bed-401e-467b-bac8-4778d8d4746c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
442025-07-15T21:15:07.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9d3c4bed-401e-467b-bac8-4778d8d4746c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
452025-07-15T21:15:07.248ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1824 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9d3c4bed-401e-467b-bac8-4778d8d4746c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
462025-07-15T21:15:08.264ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6f0b9087-6e87-4434-87e3-10587826e0ac roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
472025-07-15T21:15:08.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6f0b9087-6e87-4434-87e3-10587826e0ac resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
482025-07-15T21:15:08.266ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2313 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6f0b9087-6e87-4434-87e3-10587826e0ac response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
492025-07-15T21:15:09.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cefce41b-aaa8-4266-8181-ecef19676d40 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
502025-07-15T21:15:09.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cefce41b-aaa8-4266-8181-ecef19676d40 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
512025-07-15T21:15:09.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2318 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cefce41b-aaa8-4266-8181-ecef19676d40 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
522025-07-15T21:15:09.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 880 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 4c1cdd7d-50df-4607-94b0-73059bc2094a response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
532025-07-15T21:15:10.296ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 156d0828-224c-47a0-9561-43b0ca6387e5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
542025-07-15T21:15:10.296ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 156d0828-224c-47a0-9561-43b0ca6387e5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
552025-07-15T21:15:10.298ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2171 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 156d0828-224c-47a0-9561-43b0ca6387e5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
562025-07-15T21:15:11.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d2746630-6db3-4c3d-acbc-7ab7e3dee84e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
572025-07-15T21:15:11.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d2746630-6db3-4c3d-acbc-7ab7e3dee84e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
582025-07-15T21:15:11.315ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2512 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d2746630-6db3-4c3d-acbc-7ab7e3dee84e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
592025-07-15T21:15:11.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 108 reason = Timeout
602025-07-15T21:15:11.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
612025-07-15T21:15:11.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
622025-07-15T21:15:11.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
632025-07-15T21:15:11.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
642025-07-15T21:15:11.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
652025-07-15T21:15:11.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
662025-07-15T21:15:11.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
672025-07-15T21:15:11.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.738923ms iteration = 108 status = {"chicken_switches_updated":false}
682025-07-15T21:15:12.328ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4920ba6b-815e-425c-93d4-c4f554366a7b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
692025-07-15T21:15:12.329ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4920ba6b-815e-425c-93d4-c4f554366a7b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
702025-07-15T21:15:12.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4920ba6b-815e-425c-93d4-c4f554366a7b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
712025-07-15T21:15:13.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e088d8b-c5cd-4bb8-9c4d-d525aa58099e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
722025-07-15T21:15:13.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e088d8b-c5cd-4bb8-9c4d-d525aa58099e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
732025-07-15T21:15:13.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e088d8b-c5cd-4bb8-9c4d-d525aa58099e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
742025-07-15T21:15:14.360ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dd0a0a2f-ba5c-48c1-a82b-afd2a4d5146d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
752025-07-15T21:15:14.360ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dd0a0a2f-ba5c-48c1-a82b-afd2a4d5146d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
762025-07-15T21:15:14.362ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2553 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dd0a0a2f-ba5c-48c1-a82b-afd2a4d5146d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
772025-07-15T21:15:15.375ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2aba4125-7130-4e2c-a365-e8ba0c9f12fa roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
782025-07-15T21:15:15.376ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2aba4125-7130-4e2c-a365-e8ba0c9f12fa resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
792025-07-15T21:15:15.377ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2aba4125-7130-4e2c-a365-e8ba0c9f12fa response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
802025-07-15T21:15:16.390ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3dcffcd8-0b42-477b-a7fe-43484b6f1066 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
812025-07-15T21:15:16.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3dcffcd8-0b42-477b-a7fe-43484b6f1066 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
822025-07-15T21:15:16.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3dcffcd8-0b42-477b-a7fe-43484b6f1066 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
832025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
842025-07-15T21:15:16.816ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225
852025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.981\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225"}
862025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
872025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
882025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
892025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
902025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
912025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
922025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
932025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
942025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
952025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
962025-07-15T21:15:16.816ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-07-15 21:05:16.816167947 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
972025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
982025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
992025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
1002025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
1012025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1022025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1032025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
1042025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1052025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1062025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1072025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1082025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1092025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1102025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1112025-07-15T21:15:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1122025-07-15T21:15:16.816ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4fcf02c2-2876-4bd3-b6eb-3250110a4c40.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 306280976 } })
1132025-07-15T21:15:16.816ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ada38bf-5331-48b0-9340-612e7eca7fae.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1fb7ee6e-8db0-4006-be77-f524ca0c97d3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 306281865 } })
1142025-07-15T21:15:16.817ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
1152025-07-15T21:15:16.817ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.227302ms iteration = 10 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
1162025-07-15T21:15:16.817ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1172025-07-15T21:15:16.817ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.272623ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
1182025-07-15T21:15:16.818ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1192025-07-15T21:15:16.820ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 309413266 } })
1202025-07-15T21:15:16.820ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1212025-07-15T21:15:16.820ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.912511ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1222025-07-15T21:15:16.820ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
1232025-07-15T21:15:16.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1242025-07-15T21:15:16.820ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1252025-07-15T21:15:16.820ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-07-15 21:05:16.816167947 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1262025-07-15T21:15:16.820ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.600339ms iteration = 10 status = {"expiration":"2025-07-15T21:05:16.816167947Z","pruned":[]}
1272025-07-15T21:15:16.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1282025-07-15T21:15:16.820ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 310219737 } })
1292025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1302025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1312025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
1322025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Timeout
1332025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
1342025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 5.5\u{b5}s iteration = 13 status = "Disabled"
1352025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
1362025-07-15T21:15:16.821ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
1372025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1382025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1392025-07-15T21:15:16.821ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
1402025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1412025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1422025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1432025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1442025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1452025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1462025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1472025-07-15T21:15:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1482025-07-15T21:15:16.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1492025-07-15T21:15:16.822ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1502025-07-15T21:15:16.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1512025-07-15T21:15:16.822ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1522025-07-15T21:15:16.823ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532025-07-15T21:15:16.823ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1542025-07-15T21:15:16.823ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1552025-07-15T21:15:16.824ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1562025-07-15T21:15:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1572025-07-15T21:15:16.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1582025-07-15T21:15:16.827ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1592025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1602025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1612025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1622025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1632025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1642025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1652025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1662025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1672025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1682025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1692025-07-15T21:15:16.828ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1702025-07-15T21:15:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1712025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1722025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
1732025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1742025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
1752025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1762025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1772025-07-15T21:15:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1782025-07-15T21:15:16.831ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1792025-07-15T21:15:16.831ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1802025-07-15T21:15:16.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1812025-07-15T21:15:16.831ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1822025-07-15T21:15:16.833ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
1832025-07-15T21:15:16.833ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1842025-07-15T21:15:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1852025-07-15T21:15:16.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1862025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1872025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1882025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1892025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
1902025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
1912025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
1922025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1932025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1942025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1952025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1962025-07-15T21:15:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
1972025-07-15T21:15:16.835ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1982025-07-15T21:15:16.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1992025-07-15T21:15:16.835ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2002025-07-15T21:15:16.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2012025-07-15T21:15:16.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2022025-07-15T21:15:16.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2032025-07-15T21:15:16.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2042025-07-15T21:15:16.838ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2052025-07-15T21:15:16.838ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2062025-07-15T21:15:16.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2072025-07-15T21:15:16.838ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2082025-07-15T21:15:16.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2092025-07-15T21:15:16.840ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2102025-07-15T21:15:16.840ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.824933ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
2112025-07-15T21:15:16.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2122025-07-15T21:15:16.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2132025-07-15T21:15:16.841ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
2142025-07-15T21:15:16.841ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 7.215616ms iteration = 10 status = {"by_rx":{},"error":null}
2152025-07-15T21:15:16.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2162025-07-15T21:15:16.854ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2172025-07-15T21:15:16.854ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2182025-07-15T21:15:16.854ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 0 version = 2
2192025-07-15T21:15:16.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2202025-07-15T21:15:16.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2212025-07-15T21:15:16.856ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 35.627111ms iteration = 5 status = {"nsuccess":5}
2222025-07-15T21:15:16.856ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2232025-07-15T21:15:16.856ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2242025-07-15T21:15:16.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2252025-07-15T21:15:16.857ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
2262025-07-15T21:15:16.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
2272025-07-15T21:15:16.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2282025-07-15T21:15:16.860ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 26.235368ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"162.0.0","globs_reprocessed":{},"no_receivers":[]}
2292025-07-15T21:15:16.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2302025-07-15T21:15:16.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2312025-07-15T21:15:16.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2322025-07-15T21:15:16.866ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 4 version = 2
2332025-07-15T21:15:16.866ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2342025-07-15T21:15:16.866ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-07-15 21:06:19.267830 UTC dns_group = external generation = 2
2352025-07-15T21:15:16.866ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 50.378511ms iteration = 10 status = {"generation":2}
2362025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
2372025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
2382025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2392025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
2402025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2412025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2422025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
2432025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2442025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
2452025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
2462025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2472025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
2482025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2492025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
2502025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2512025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
2522025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
2532025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2542025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2552025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 109 reason = Timeout
2562025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
2572025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2582025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2592025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2602025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2612025-07-15T21:15:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2622025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2632025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2642025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2652025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
2662025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2672025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2682025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
2692025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2702025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2712025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2722025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
2732025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
2742025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2752025-07-15T21:15:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2762025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
2772025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
2782025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2792025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
2802025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2812025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
2822025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2832025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
2842025-07-15T21:15:16.888ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2852025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2862025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
2872025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2882025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2892025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2902025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2912025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2922025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2932025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2942025-07-15T21:15:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2952025-07-15T21:15:16.888ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 378201194 } })
2962025-07-15T21:15:16.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2972025-07-15T21:15:16.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2982025-07-15T21:15:16.893ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2992025-07-15T21:15:16.893ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
3002025-07-15T21:15:16.893ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 72.677608ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3012025-07-15T21:15:16.896ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3022025-07-15T21:15:16.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3032025-07-15T21:15:16.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3042025-07-15T21:15:16.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3052025-07-15T21:15:16.899ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
3062025-07-15T21:15:16.899ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
3072025-07-15T21:15:16.899ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 69.008373ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
3082025-07-15T21:15:16.900ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3092025-07-15T21:15:16.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3102025-07-15T21:15:16.903ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 16.623629ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
3112025-07-15T21:15:16.904ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3122025-07-15T21:15:16.904ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3132025-07-15T21:15:16.905ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
3142025-07-15T21:15:16.905ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
3152025-07-15T21:15:16.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
3162025-07-15T21:15:16.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3172025-07-15T21:15:16.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3182025-07-15T21:15:16.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3192025-07-15T21:15:16.907ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3202025-07-15T21:15:16.909ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3212025-07-15T21:15:16.909ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3222025-07-15T21:15:16.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3232025-07-15T21:15:16.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3242025-07-15T21:15:16.910ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3252025-07-15T21:15:16.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3262025-07-15T21:15:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3272025-07-15T21:15:16.912ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3282025-07-15T21:15:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3292025-07-15T21:15:16.915ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.494312ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3302025-07-15T21:15:16.917ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
3312025-07-15T21:15:16.917ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 31.050302ms iteration = 109 status = {"chicken_switches_updated":false}
3322025-07-15T21:15:16.918ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3332025-07-15T21:15:16.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3342025-07-15T21:15:16.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3352025-07-15T21:15:16.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3362025-07-15T21:15:16.929ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3372025-07-15T21:15:16.929ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3382025-07-15T21:15:16.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3392025-07-15T21:15:16.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3402025-07-15T21:15:16.932ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3412025-07-15T21:15:16.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3422025-07-15T21:15:16.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3432025-07-15T21:15:16.934ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
3442025-07-15T21:15:16.934ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3452025-07-15T21:15:16.934ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "90f5b604-93cc-47b9-b742-69f400e4e7db", "content-length": "26", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
3462025-07-15T21:15:16.934ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3472025-07-15T21:15:16.936ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-07-15T21:06:12.596852Z, time_modified: 2025-07-15T21:06:12.596852Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }) }
3482025-07-15T21:15:16.936ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 120.476173ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]}},"default_endpoint":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3492025-07-15T21:15:16.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
3502025-07-15T21:15:16.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3512025-07-15T21:15:16.938ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3522025-07-15T21:15:16.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3532025-07-15T21:15:16.964ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
3542025-07-15T21:15:16.964ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.742898ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
3552025-07-15T21:15:16.972ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3562025-07-15T21:15:16.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3572025-07-15T21:15:16.975ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3582025-07-15T21:15:16.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3592025-07-15T21:15:16.977ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3602025-07-15T21:15:16.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3612025-07-15T21:15:16.978ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1524, tv_nsec: 468157018 } })
3622025-07-15T21:15:16.979ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
3632025-07-15T21:15:16.981ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3642025-07-15T21:15:16.981ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3652025-07-15T21:15:16.981ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
3662025-07-15T21:15:16.981ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3672025-07-15T21:15:16.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3682025-07-15T21:15:16.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3692025-07-15T21:15:16.983ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3702025-07-15T21:15:16.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3712025-07-15T21:15:16.988ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 101.991916ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3722025-07-15T21:15:16.989ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 101.703788ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
3732025-07-15T21:15:16.989ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3742025-07-15T21:15:16.991ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3752025-07-15T21:15:16.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3762025-07-15T21:15:16.992ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 104.273734ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
3772025-07-15T21:15:16.992ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3782025-07-15T21:15:16.993ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3792025-07-15T21:15:16.993ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3802025-07-15T21:15:16.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3812025-07-15T21:15:16.996ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3822025-07-15T21:15:16.997ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 108.841282ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3832025-07-15T21:15:16.997ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
3842025-07-15T21:15:16.997ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3852025-07-15T21:15:16.997ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3862025-07-15T21:15:16.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3872025-07-15T21:15:17.000ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 111.824049ms iteration = 19 status = 0
3882025-07-15T21:15:17.000ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3892025-07-15T21:15:17.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3902025-07-15T21:15:17.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3912025-07-15T21:15:17.001ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3922025-07-15T21:15:17.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3932025-07-15T21:15:17.002ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3942025-07-15T21:15:17.002ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3952025-07-15T21:15:17.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3962025-07-15T21:15:17.005ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3972025-07-15T21:15:17.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3982025-07-15T21:15:17.006ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3992025-07-15T21:15:17.006ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4002025-07-15T21:15:17.007ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4012025-07-15T21:15:17.007ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 118.80146ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4022025-07-15T21:15:17.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4032025-07-15T21:15:17.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4042025-07-15T21:15:17.011ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4052025-07-15T21:15:17.011ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4062025-07-15T21:15:17.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4072025-07-15T21:15:17.013ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4082025-07-15T21:15:17.013ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4092025-07-15T21:15:17.013ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4102025-07-15T21:15:17.013ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4112025-07-15T21:15:17.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4122025-07-15T21:15:17.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4132025-07-15T21:15:17.015ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4142025-07-15T21:15:17.018ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4152025-07-15T21:15:17.018ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4162025-07-15T21:15:17.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4172025-07-15T21:15:17.022ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
4182025-07-15T21:15:17.022ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4192025-07-15T21:15:17.022ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-07-15 21:06:16.954295 UTC dns_group = internal generation = 1
4202025-07-15T21:15:17.022ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 205.916641ms iteration = 10 status = {"generation":1}
4212025-07-15T21:15:17.030ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4222025-07-15T21:15:17.048ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4232025-07-15T21:15:17.048ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4242025-07-15T21:15:17.048ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4252025-07-15T21:15:17.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4262025-07-15T21:15:17.054ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
4272025-07-15T21:15:17.054ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 223.548956ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4282025-07-15T21:15:17.064ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
4292025-07-15T21:15:17.064ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 177.981989ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
4302025-07-15T21:15:17.070ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 182.327261ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4312025-07-15T21:15:17.082ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4322025-07-15T21:15:17.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4332025-07-15T21:15:17.085ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4342025-07-15T21:15:17.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4352025-07-15T21:15:17.087ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4362025-07-15T21:15:17.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4372025-07-15T21:15:17.089ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4382025-07-15T21:15:17.091ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4392025-07-15T21:15:17.091ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4402025-07-15T21:15:17.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4412025-07-15T21:15:17.099ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
4422025-07-15T21:15:17.099ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4432025-07-15T21:15:17.100ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "76f1c109-cfb8-4d8e-b4a4-0bf07db2667a", "content-length": "26", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
4442025-07-15T21:15:17.100ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4452025-07-15T21:15:17.102ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4462025-07-15T21:15:17.103ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4472025-07-15T21:15:17.103ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4482025-07-15T21:15:17.103ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 217.320465ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4492025-07-15T21:15:17.156ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
4502025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a16f922-c281-4a31-a70b-de7d5e3317cc", "content-length": "78", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
4512025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
4522025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
4532025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
4542025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
4552025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
4562025-07-15T21:15:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
4572025-07-15T21:15:17.159ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
4582025-07-15T21:15:17.159ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
4592025-07-15T21:15:17.160ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
4602025-07-15T21:15:17.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
4612025-07-15T21:15:17.162ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
4622025-07-15T21:15:17.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
4632025-07-15T21:15:17.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
4642025-07-15T21:15:17.163ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
4652025-07-15T21:15:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
4662025-07-15T21:15:17.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
4672025-07-15T21:15:17.163ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
4682025-07-15T21:15:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
4692025-07-15T21:15:17.164ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 278.27398ms iteration = 21 status = {"physical_disks_added":0}
4702025-07-15T21:15:17.164ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
4712025-07-15T21:15:17.164ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
4722025-07-15T21:15:17.165ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "83539cc4-e027-4769-a958-f05657ec5e12", "content-length": "81", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
4732025-07-15T21:15:17.175ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
4742025-07-15T21:15:17.175ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4752025-07-15T21:15:17.176ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "199beaf0-a588-4893-b299-9e6889bb3b45", "content-length": "26", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
4762025-07-15T21:15:17.176ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4772025-07-15T21:15:17.177ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
4782025-07-15T21:15:17.202ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
4792025-07-15T21:15:17.202ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
4802025-07-15T21:15:17.202ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "24c25468-c9ad-4ea0-9e67-08fb09ff20b6", "content-length": "81", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
4812025-07-15T21:15:17.208ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
4822025-07-15T21:15:17.208ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
4832025-07-15T21:15:17.208ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ba4f075-e1b8-4457-8679-c4a572edc448", "content-length": "81", "date": "Tue, 15 Jul 2025 21:15:16 GMT"} })
4842025-07-15T21:15:17.210ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 322.259717ms iteration = 19 status = {"errors":[],"sps":[]}
4852025-07-15T21:15:17.229ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4862025-07-15T21:15:17.337ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 521.419973ms iteration = 19 status = {}
4872025-07-15T21:15:17.344ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4882025-07-15T21:15:17.344ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "aa0be153-e335-49a8-8c83-4b4c9ef69742", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4892025-07-15T21:15:17.344ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4902025-07-15T21:15:17.348ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a65478fe-2400-4f81-9b1f-21727ac538eb", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4912025-07-15T21:15:17.348ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4922025-07-15T21:15:17.350ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4932025-07-15T21:15:17.350ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 534.265815ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4942025-07-15T21:15:17.352ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4952025-07-15T21:15:17.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4962025-07-15T21:15:17.361ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 545.188907ms iteration = 19 status = null
4972025-07-15T21:15:17.408ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a8ae3f48-fd95-4dc7-9511-ac1049c2be1d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
4982025-07-15T21:15:17.408ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a8ae3f48-fd95-4dc7-9511-ac1049c2be1d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
4992025-07-15T21:15:17.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3386 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a8ae3f48-fd95-4dc7-9511-ac1049c2be1d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5002025-07-15T21:15:17.438ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5012025-07-15T21:15:17.438ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "488d3f1a-2672-4bc8-9c3b-989e9292bac3", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5022025-07-15T21:15:17.438ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5032025-07-15T21:15:17.441ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4fbb8b23-5bda-4e30-b256-6ceb258ea748", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5042025-07-15T21:15:17.441ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5052025-07-15T21:15:17.443ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e7acf86c-5182-4f8a-8520-3ab1db311b5c", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5062025-07-15T21:15:17.443ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5072025-07-15T21:15:17.445ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
5082025-07-15T21:15:17.445ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 629.130566ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5092025-07-15T21:15:17.456ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5102025-07-15T21:15:17.456ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5112025-07-15T21:15:17.457ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "96e2e46b-115b-43c5-ae52-5962a8349c40", "content-length": "372", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} })
5122025-07-15T21:15:17.457ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5132025-07-15T21:15:17.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5142025-07-15T21:15:17.458ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5152025-07-15T21:15:17.459ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8820f04-37f1-4120-a8a3-b9f125724b27", "content-length": "691", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} })
5162025-07-15T21:15:17.459ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5172025-07-15T21:15:17.459ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5182025-07-15T21:15:17.459ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5192025-07-15T21:15:17.461ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 575.12394ms iteration = 19 status = {}
5202025-07-15T21:15:17.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5212025-07-15T21:15:17.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5222025-07-15T21:15:17.506ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
5232025-07-15T21:15:17.506ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5242025-07-15T21:15:17.507ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
5252025-07-15T21:15:17.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5262025-07-15T21:15:17.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5272025-07-15T21:15:17.512ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9502be0c-51b6-4f70-9cf2-7b5e39181dd8", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} })
5282025-07-15T21:15:17.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5292025-07-15T21:15:17.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5302025-07-15T21:15:17.514ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5312025-07-15T21:15:17.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5322025-07-15T21:15:17.523ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
5332025-07-15T21:15:17.523ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
5342025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
5352025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
5362025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5372025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5382025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5392025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5402025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
5412025-07-15T21:15:17.524ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
5422025-07-15T21:15:17.524ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5432025-07-15T21:15:17.525ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
5442025-07-15T21:15:17.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5452025-07-15T21:15:17.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5462025-07-15T21:15:17.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5472025-07-15T21:15:17.528ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
5482025-07-15T21:15:17.528ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5492025-07-15T21:15:17.528ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5502025-07-15T21:15:17.528ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5512025-07-15T21:15:17.532ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a5f018c3-0e02-47ef-9614-2ec9fedbf9c0", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} })
5522025-07-15T21:15:17.535ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 646.863375ms iteration = 19 status = {}
5532025-07-15T21:15:17.742ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0c21a7f5-4e4b-4010-afe4-b0aabad64475", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} })
5542025-07-15T21:15:17.742ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
5552025-07-15T21:15:17.742ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5562025-07-15T21:15:17.742ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e04701c5-8512-4fe7-a370-7c5827d630be", "content-length": "682", "date": "Tue, 15 Jul 2025 21:15:17 GMT"} })
5572025-07-15T21:15:17.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5582025-07-15T21:15:17.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5592025-07-15T21:15:17.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5602025-07-15T21:15:17.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5612025-07-15T21:15:17.749ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5622025-07-15T21:15:17.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5632025-07-15T21:15:17.752ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5642025-07-15T21:15:17.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5652025-07-15T21:15:17.760ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 874.206999ms iteration = 19 status = {}
5662025-07-15T21:15:18.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f1daac5e-7be6-4fce-b4ff-feee1718d36c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5672025-07-15T21:15:18.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f1daac5e-7be6-4fce-b4ff-feee1718d36c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5682025-07-15T21:15:18.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f1daac5e-7be6-4fce-b4ff-feee1718d36c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5692025-07-15T21:15:19.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a89ab93a-50b5-4a03-9e5a-59de0a7b50ef roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5702025-07-15T21:15:19.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a89ab93a-50b5-4a03-9e5a-59de0a7b50ef resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5712025-07-15T21:15:19.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2370 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a89ab93a-50b5-4a03-9e5a-59de0a7b50ef response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5722025-07-15T21:15:19.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 500 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = dd316576-c36f-4760-ae0a-14d2f3e0d673 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
5732025-07-15T21:15:20.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d0cd7002-b208-4f12-a6e1-b368fba54425 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5742025-07-15T21:15:20.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d0cd7002-b208-4f12-a6e1-b368fba54425 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5752025-07-15T21:15:20.462ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2110 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d0cd7002-b208-4f12-a6e1-b368fba54425 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5762025-07-15T21:15:21.480ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f4482d6a-e8e1-4377-bc3a-5d540ca6491d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5772025-07-15T21:15:21.480ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f4482d6a-e8e1-4377-bc3a-5d540ca6491d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5782025-07-15T21:15:21.482ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2271 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f4482d6a-e8e1-4377-bc3a-5d540ca6491d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5792025-07-15T21:15:21.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 110 reason = Timeout
5802025-07-15T21:15:21.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5812025-07-15T21:15:21.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5822025-07-15T21:15:21.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5832025-07-15T21:15:21.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
5842025-07-15T21:15:21.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5852025-07-15T21:15:21.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5862025-07-15T21:15:21.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
5872025-07-15T21:15:21.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.593041ms iteration = 110 status = {"chicken_switches_updated":false}
5882025-07-15T21:15:22.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fdbf2432-432e-46d2-a979-d784f9d4518f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5892025-07-15T21:15:22.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fdbf2432-432e-46d2-a979-d784f9d4518f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5902025-07-15T21:15:22.498ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2256 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fdbf2432-432e-46d2-a979-d784f9d4518f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5912025-07-15T21:15:23.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 86d941f9-8923-4ac6-b4b0-5f3b47bd7289 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5922025-07-15T21:15:23.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 86d941f9-8923-4ac6-b4b0-5f3b47bd7289 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5932025-07-15T21:15:23.514ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2360 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 86d941f9-8923-4ac6-b4b0-5f3b47bd7289 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5942025-07-15T21:15:24.529ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c145957d-db52-4479-8c71-8deda6963be3 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5952025-07-15T21:15:24.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c145957d-db52-4479-8c71-8deda6963be3 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5962025-07-15T21:15:24.531ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2095 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c145957d-db52-4479-8c71-8deda6963be3 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5972025-07-15T21:15:25.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3d749e56-b04e-4fa6-88a8-38f080495521 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5982025-07-15T21:15:25.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3d749e56-b04e-4fa6-88a8-38f080495521 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
5992025-07-15T21:15:25.548ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2840 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3d749e56-b04e-4fa6-88a8-38f080495521 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6002025-07-15T21:15:26.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = efdb6809-9f1d-4ca9-9560-f11db5e1a10e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6012025-07-15T21:15:26.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = efdb6809-9f1d-4ca9-9560-f11db5e1a10e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6022025-07-15T21:15:26.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1798 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = efdb6809-9f1d-4ca9-9560-f11db5e1a10e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6032025-07-15T21:15:26.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
6042025-07-15T21:15:26.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6052025-07-15T21:15:26.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
6062025-07-15T21:15:26.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 111 reason = Timeout
6072025-07-15T21:15:26.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6082025-07-15T21:15:26.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6092025-07-15T21:15:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6102025-07-15T21:15:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6112025-07-15T21:15:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6122025-07-15T21:15:26.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6132025-07-15T21:15:26.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142025-07-15T21:15:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
6152025-07-15T21:15:26.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6162025-07-15T21:15:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6172025-07-15T21:15:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6182025-07-15T21:15:26.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6192025-07-15T21:15:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6202025-07-15T21:15:26.895ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
6212025-07-15T21:15:26.895ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.708085ms iteration = 111 status = {"chicken_switches_updated":false}
6222025-07-15T21:15:26.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6232025-07-15T21:15:26.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6242025-07-15T21:15:26.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6252025-07-15T21:15:26.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
6262025-07-15T21:15:26.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6272025-07-15T21:15:26.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6282025-07-15T21:15:26.909ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
6292025-07-15T21:15:26.909ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.504771ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
6302025-07-15T21:15:26.923ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6312025-07-15T21:15:26.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6322025-07-15T21:15:26.966ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 80.001677ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6332025-07-15T21:15:27.578ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = aba696ef-1721-4ff6-9af2-c8184722c712 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6342025-07-15T21:15:27.579ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = aba696ef-1721-4ff6-9af2-c8184722c712 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6352025-07-15T21:15:27.581ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2748 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = aba696ef-1721-4ff6-9af2-c8184722c712 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6362025-07-15T21:15:28.596ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 29215add-dd26-4087-b79a-f4e2efe0db57 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6372025-07-15T21:15:28.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 29215add-dd26-4087-b79a-f4e2efe0db57 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6382025-07-15T21:15:28.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2142 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 29215add-dd26-4087-b79a-f4e2efe0db57 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6392025-07-15T21:15:29.612ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3c119485-590e-42bc-88b0-65e520fae205 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6402025-07-15T21:15:29.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3c119485-590e-42bc-88b0-65e520fae205 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6412025-07-15T21:15:29.615ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3c119485-590e-42bc-88b0-65e520fae205 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6422025-07-15T21:15:29.968ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:42847
6432025-07-15T21:15:29.968ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = dd0e832d-4b8c-476f-ab9f-315dcbf8b69f roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
6442025-07-15T21:15:29.968ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = dd0e832d-4b8c-476f-ab9f-315dcbf8b69f resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
6452025-07-15T21:15:29.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2809 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = dd0e832d-4b8c-476f-ab9f-315dcbf8b69f response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
6462025-07-15T21:15:29.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 2dbf3a28-3553-48bd-ab2c-1a843b953815 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
6472025-07-15T21:15:29.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 2dbf3a28-3553-48bd-ab2c-1a843b953815 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
6482025-07-15T21:15:29.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2713 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 2dbf3a28-3553-48bd-ab2c-1a843b953815 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
6492025-07-15T21:15:29.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 456 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 4e22e013-e3ce-495e-ab04-9d896ba780bf response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
6502025-07-15T21:15:30.633ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 87261b52-4458-4c42-a2c8-c1dc7503b080 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6512025-07-15T21:15:30.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 87261b52-4458-4c42-a2c8-c1dc7503b080 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6522025-07-15T21:15:30.635ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 87261b52-4458-4c42-a2c8-c1dc7503b080 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6532025-07-15T21:15:31.649ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1f48139f-4ac3-4c33-b31a-f9a177f79fba roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6542025-07-15T21:15:31.650ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1f48139f-4ac3-4c33-b31a-f9a177f79fba resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6552025-07-15T21:15:31.652ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2363 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1f48139f-4ac3-4c33-b31a-f9a177f79fba response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6562025-07-15T21:15:31.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 112 reason = Timeout
6572025-07-15T21:15:31.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6582025-07-15T21:15:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6592025-07-15T21:15:31.929ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6602025-07-15T21:15:31.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
6612025-07-15T21:15:31.932ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6622025-07-15T21:15:31.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6632025-07-15T21:15:31.934ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
6642025-07-15T21:15:31.934ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 47.694183ms iteration = 112 status = {"chicken_switches_updated":false}
6652025-07-15T21:15:32.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = de44f030-efa5-407d-a5d4-0fe350c52307 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6662025-07-15T21:15:32.666ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = de44f030-efa5-407d-a5d4-0fe350c52307 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6672025-07-15T21:15:32.668ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2266 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = de44f030-efa5-407d-a5d4-0fe350c52307 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6682025-07-15T21:15:33.681ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 54007cdc-f4d1-4881-a902-24bcd4d0d46d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6692025-07-15T21:15:33.682ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 54007cdc-f4d1-4881-a902-24bcd4d0d46d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6702025-07-15T21:15:33.683ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2073 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 54007cdc-f4d1-4881-a902-24bcd4d0d46d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6712025-07-15T21:15:34.698ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3bcf309b-c3a3-4da5-bd5a-62f42bc00867 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6722025-07-15T21:15:34.698ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3bcf309b-c3a3-4da5-bd5a-62f42bc00867 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6732025-07-15T21:15:34.699ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1926 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3bcf309b-c3a3-4da5-bd5a-62f42bc00867 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6742025-07-15T21:15:35.713ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4d66a753-8c79-4328-b81c-3a35f3225e38 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6752025-07-15T21:15:35.713ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4d66a753-8c79-4328-b81c-3a35f3225e38 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6762025-07-15T21:15:35.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2664 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4d66a753-8c79-4328-b81c-3a35f3225e38 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6772025-07-15T21:15:36.730ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c14a2318-080a-4370-8960-1ddefb780fc1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6782025-07-15T21:15:36.730ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c14a2318-080a-4370-8960-1ddefb780fc1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6792025-07-15T21:15:36.732ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2162 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c14a2318-080a-4370-8960-1ddefb780fc1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
6802025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
6812025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
6822025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6832025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6842025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 113 reason = Timeout
6852025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6862025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6872025-07-15T21:15:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6882025-07-15T21:15:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6892025-07-15T21:15:36.888ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6902025-07-15T21:15:36.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6912025-07-15T21:15:36.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6922025-07-15T21:15:36.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932025-07-15T21:15:36.890ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.490431ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6942025-07-15T21:15:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6952025-07-15T21:15:36.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6962025-07-15T21:15:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6972025-07-15T21:15:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
6982025-07-15T21:15:36.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6992025-07-15T21:15:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
7002025-07-15T21:15:36.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7012025-07-15T21:15:36.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
7022025-07-15T21:15:36.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.257048ms iteration = 113 status = {"chicken_switches_updated":false}
7032025-07-15T21:15:36.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7042025-07-15T21:15:36.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7052025-07-15T21:15:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
7062025-07-15T21:15:36.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7072025-07-15T21:15:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7082025-07-15T21:15:36.910ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
7092025-07-15T21:15:36.910ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.767334ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
7102025-07-15T21:15:37.746ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9a1bbb6c-55b1-4de3-a3f1-3dd77a9aa4bb roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7112025-07-15T21:15:37.746ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9a1bbb6c-55b1-4de3-a3f1-3dd77a9aa4bb resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7122025-07-15T21:15:37.748ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2442 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9a1bbb6c-55b1-4de3-a3f1-3dd77a9aa4bb response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7132025-07-15T21:15:38.762ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 08bf4596-50a3-4593-92b0-8170b5124d9c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7142025-07-15T21:15:38.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 08bf4596-50a3-4593-92b0-8170b5124d9c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7152025-07-15T21:15:38.764ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2123 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 08bf4596-50a3-4593-92b0-8170b5124d9c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7162025-07-15T21:15:39.779ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c715351d-ebbc-481e-a4fa-6f14b0786c15 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7172025-07-15T21:15:39.780ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c715351d-ebbc-481e-a4fa-6f14b0786c15 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7182025-07-15T21:15:39.781ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2007 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c715351d-ebbc-481e-a4fa-6f14b0786c15 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7192025-07-15T21:15:39.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 507 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = c625f417-7e08-42f8-b817-231e4443760e response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
7202025-07-15T21:15:40.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d4610b3d-47fd-4496-a8f2-dca87f1079b0 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7212025-07-15T21:15:40.796ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d4610b3d-47fd-4496-a8f2-dca87f1079b0 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7222025-07-15T21:15:40.798ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d4610b3d-47fd-4496-a8f2-dca87f1079b0 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7232025-07-15T21:15:41.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2486757f-0e15-4b50-b689-9b3b26619038 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7242025-07-15T21:15:41.813ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2486757f-0e15-4b50-b689-9b3b26619038 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7252025-07-15T21:15:41.814ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1961 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2486757f-0e15-4b50-b689-9b3b26619038 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7262025-07-15T21:15:41.887ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 114 reason = Timeout
7272025-07-15T21:15:41.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
7282025-07-15T21:15:41.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
7292025-07-15T21:15:41.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7302025-07-15T21:15:41.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
7312025-07-15T21:15:41.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
7322025-07-15T21:15:41.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
7332025-07-15T21:15:41.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
7342025-07-15T21:15:41.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.819326ms iteration = 114 status = {"chicken_switches_updated":false}
7352025-07-15T21:15:42.828ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 870532f6-a04f-4907-b049-036c533fb5f0 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7362025-07-15T21:15:42.829ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 870532f6-a04f-4907-b049-036c533fb5f0 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7372025-07-15T21:15:42.831ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2162 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 870532f6-a04f-4907-b049-036c533fb5f0 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7382025-07-15T21:15:43.845ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dcce4e33-86f2-4108-a5a0-0bf5df61facf roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7392025-07-15T21:15:43.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dcce4e33-86f2-4108-a5a0-0bf5df61facf resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7402025-07-15T21:15:43.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2135 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dcce4e33-86f2-4108-a5a0-0bf5df61facf response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7412025-07-15T21:15:44.862ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca2a6884-6c3a-4755-9efa-5d13278fa1df roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7422025-07-15T21:15:44.862ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca2a6884-6c3a-4755-9efa-5d13278fa1df resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7432025-07-15T21:15:44.864ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2686 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca2a6884-6c3a-4755-9efa-5d13278fa1df response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7442025-07-15T21:15:45.879ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 54443abc-3124-4e15-8a6d-f5cf57ed3226 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7452025-07-15T21:15:45.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 54443abc-3124-4e15-8a6d-f5cf57ed3226 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7462025-07-15T21:15:45.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2247 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 54443abc-3124-4e15-8a6d-f5cf57ed3226 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
7472025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
7482025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
7492025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
7502025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7512025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7522025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7532025-07-15T21:15:46.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7542025-07-15T21:15:46.817ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 1.891189ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7552025-07-15T21:15:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 307352493 } })
7562025-07-15T21:15:46.818ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
7572025-07-15T21:15:46.818ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 308120793 } })
7582025-07-15T21:15:46.818ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
7592025-07-15T21:15:46.885ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
7602025-07-15T21:15:46.885ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7612025-07-15T21:15:46.885ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7f61cea-7c10-4c11-abc1-7a4b20b712d9", "content-length": "26", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
7622025-07-15T21:15:46.885ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7632025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
7642025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
7652025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
7662025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
7672025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7682025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7692025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
7702025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7712025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7722025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
7732025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7742025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7752025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7762025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
7772025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7782025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7792025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
7802025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7812025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7822025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
7832025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7842025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7852025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
7862025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7872025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
7882025-07-15T21:15:46.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7892025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
7902025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 115 reason = Timeout
7912025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7922025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
7932025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
7942025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7952025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7962025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
7972025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7982025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7992025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
8002025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8012025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
8022025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8032025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
8042025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8052025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
8062025-07-15T21:15:46.890ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8072025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8082025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
8092025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8102025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8112025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8122025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8132025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8142025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8152025-07-15T21:15:46.890ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 380111753 } })
8162025-07-15T21:15:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8172025-07-15T21:15:46.891ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8182025-07-15T21:15:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8192025-07-15T21:15:46.893ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
8202025-07-15T21:15:46.893ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8212025-07-15T21:15:46.895ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a9eba9b-0ba3-4747-b51f-a8613b6cd0dc", "content-length": "26", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
8222025-07-15T21:15:46.895ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8232025-07-15T21:15:46.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d4fca06d-3e6e-4616-ba7e-82c55a1cf85e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
8242025-07-15T21:15:46.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d4fca06d-3e6e-4616-ba7e-82c55a1cf85e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
8252025-07-15T21:15:46.896ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8262025-07-15T21:15:46.898ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 11.379434ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
8272025-07-15T21:15:46.898ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8282025-07-15T21:15:46.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8292025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8302025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
8312025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8322025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8332025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8342025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8352025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8362025-07-15T21:15:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8372025-07-15T21:15:46.902ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8382025-07-15T21:15:46.902ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8392025-07-15T21:15:46.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8402025-07-15T21:15:46.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8412025-07-15T21:15:46.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422025-07-15T21:15:46.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8432025-07-15T21:15:46.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8442025-07-15T21:15:46.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8452025-07-15T21:15:46.906ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8462025-07-15T21:15:46.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 11208 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d4fca06d-3e6e-4616-ba7e-82c55a1cf85e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
8472025-07-15T21:15:46.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8482025-07-15T21:15:46.907ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8492025-07-15T21:15:46.907ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8502025-07-15T21:15:46.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8512025-07-15T21:15:46.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8522025-07-15T21:15:46.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8532025-07-15T21:15:46.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8542025-07-15T21:15:46.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
8552025-07-15T21:15:46.908ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8562025-07-15T21:15:46.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8572025-07-15T21:15:46.909ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8582025-07-15T21:15:46.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8592025-07-15T21:15:46.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8602025-07-15T21:15:46.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8612025-07-15T21:15:46.910ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1554, tv_nsec: 400597532 } })
8622025-07-15T21:15:46.911ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8632025-07-15T21:15:46.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8642025-07-15T21:15:46.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8652025-07-15T21:15:46.911ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
8662025-07-15T21:15:46.911ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
8672025-07-15T21:15:46.911ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 22.728977ms iteration = 115 status = {"chicken_switches_updated":false}
8682025-07-15T21:15:46.912ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8692025-07-15T21:15:46.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8702025-07-15T21:15:46.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8712025-07-15T21:15:46.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8722025-07-15T21:15:46.915ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8732025-07-15T21:15:46.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8742025-07-15T21:15:46.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8752025-07-15T21:15:46.916ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
8762025-07-15T21:15:46.916ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 27.204343ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
8772025-07-15T21:15:46.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
8782025-07-15T21:15:46.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8792025-07-15T21:15:46.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8802025-07-15T21:15:46.953ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8812025-07-15T21:15:46.953ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
8822025-07-15T21:15:46.954ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.868667ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
8832025-07-15T21:15:46.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8842025-07-15T21:15:46.954ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.660646ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
8852025-07-15T21:15:46.967ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8862025-07-15T21:15:46.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8872025-07-15T21:15:46.968ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8882025-07-15T21:15:46.968ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
8892025-07-15T21:15:46.968ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8902025-07-15T21:15:46.969ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8912025-07-15T21:15:46.969ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8922025-07-15T21:15:46.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8932025-07-15T21:15:46.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8942025-07-15T21:15:46.970ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8952025-07-15T21:15:46.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8962025-07-15T21:15:46.971ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.796073ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
8972025-07-15T21:15:46.972ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8982025-07-15T21:15:46.972ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8992025-07-15T21:15:46.972ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9002025-07-15T21:15:46.972ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9012025-07-15T21:15:46.972ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9022025-07-15T21:15:46.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9032025-07-15T21:15:46.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9042025-07-15T21:15:46.975ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9052025-07-15T21:15:46.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9062025-07-15T21:15:46.988ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9072025-07-15T21:15:46.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9082025-07-15T21:15:46.990ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 101.593985ms iteration = 22 status = {"physical_disks_added":0}
9092025-07-15T21:15:46.991ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9102025-07-15T21:15:46.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9112025-07-15T21:15:46.994ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9122025-07-15T21:15:46.997ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
9132025-07-15T21:15:46.997ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9142025-07-15T21:15:46.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9152025-07-15T21:15:46.999ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 110.272769ms iteration = 20 status = 0
9162025-07-15T21:15:47.013ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9172025-07-15T21:15:47.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9182025-07-15T21:15:47.017ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9192025-07-15T21:15:47.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9202025-07-15T21:15:47.020ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 205.543481ms iteration = 20 status = null
9212025-07-15T21:15:47.024ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9222025-07-15T21:15:47.025ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
9232025-07-15T21:15:47.025ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 136.763634ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
9242025-07-15T21:15:47.025ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 136.336763ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9252025-07-15T21:15:47.025ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 136.431845ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9262025-07-15T21:15:47.025ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 136.22237ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9272025-07-15T21:15:47.026ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
9282025-07-15T21:15:47.026ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9292025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "f69ace52-69c3-4ba1-be51-703745f09304", "content-length": "78", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
9302025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
9312025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
9322025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
9332025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
9342025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
9352025-07-15T21:15:47.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
9362025-07-15T21:15:47.028ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 213.320242ms iteration = 20 status = {}
9372025-07-15T21:15:47.028ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
9382025-07-15T21:15:47.029ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
9392025-07-15T21:15:47.029ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9402025-07-15T21:15:47.029ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "84fb768f-1b8b-4c0a-9353-79a53d3855c5", "content-length": "26", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
9412025-07-15T21:15:47.029ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9422025-07-15T21:15:47.029ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
9432025-07-15T21:15:47.029ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9442025-07-15T21:15:47.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9452025-07-15T21:15:47.030ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
9462025-07-15T21:15:47.030ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
9472025-07-15T21:15:47.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
9482025-07-15T21:15:47.031ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
9492025-07-15T21:15:47.058ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9502025-07-15T21:15:47.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9512025-07-15T21:15:47.058ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 171.018379ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
9522025-07-15T21:15:47.068ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
9532025-07-15T21:15:47.070ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
9542025-07-15T21:15:47.070ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
9552025-07-15T21:15:47.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
9562025-07-15T21:15:47.071ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
9572025-07-15T21:15:47.072ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
9582025-07-15T21:15:47.072ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
9592025-07-15T21:15:47.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
9602025-07-15T21:15:47.073ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
9612025-07-15T21:15:47.073ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
9622025-07-15T21:15:47.073ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef655da4-0f9d-4de0-9e1b-a1365540a520", "content-length": "81", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
9632025-07-15T21:15:47.073ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
9642025-07-15T21:15:47.073ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 185.860523ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
9652025-07-15T21:15:47.107ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9662025-07-15T21:15:47.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9672025-07-15T21:15:47.109ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
9682025-07-15T21:15:47.109ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
9692025-07-15T21:15:47.109ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2c8ef68-b84c-4cbb-9333-ab3d15c5f834", "content-length": "81", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
9702025-07-15T21:15:47.109ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9712025-07-15T21:15:47.111ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9722025-07-15T21:15:47.111ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9732025-07-15T21:15:47.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9742025-07-15T21:15:47.113ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9752025-07-15T21:15:47.115ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9762025-07-15T21:15:47.115ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9772025-07-15T21:15:47.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9782025-07-15T21:15:47.116ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
9792025-07-15T21:15:47.116ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
9802025-07-15T21:15:47.116ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "ada21797-7fad-4e67-8230-e035cd2b5a51", "content-length": "81", "date": "Tue, 15 Jul 2025 21:15:46 GMT"} })
9812025-07-15T21:15:47.118ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 228.684949ms iteration = 20 status = {"errors":[],"sps":[]}
9822025-07-15T21:15:47.216ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9832025-07-15T21:15:47.216ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "928e459e-d1d6-458a-a2e6-4fb002395da8", "content-length": "691", "date": "Tue, 15 Jul 2025 21:15:47 GMT"} })
9842025-07-15T21:15:47.216ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
9852025-07-15T21:15:47.217ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
9862025-07-15T21:15:47.217ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
9872025-07-15T21:15:47.219ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 330.818858ms iteration = 20 status = {}
9882025-07-15T21:15:47.234ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
9892025-07-15T21:15:47.234ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9902025-07-15T21:15:47.235ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "05927d9d-34ef-47e6-92e3-9f1923e9f6d4", "content-length": "372", "date": "Tue, 15 Jul 2025 21:15:47 GMT"} })
9912025-07-15T21:15:47.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9922025-07-15T21:15:47.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9932025-07-15T21:15:47.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9942025-07-15T21:15:47.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9952025-07-15T21:15:47.275ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
9962025-07-15T21:15:47.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9972025-07-15T21:15:47.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9982025-07-15T21:15:47.286ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
9992025-07-15T21:15:47.286ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
10002025-07-15T21:15:47.287ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
10012025-07-15T21:15:47.287ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
10022025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
10032025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
10042025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10052025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10062025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10072025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10082025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10092025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10102025-07-15T21:15:47.288ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
10112025-07-15T21:15:47.288ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10122025-07-15T21:15:47.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10132025-07-15T21:15:47.291ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
10142025-07-15T21:15:47.291ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
10152025-07-15T21:15:47.293ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7e25264c-ca12-4623-971e-7c9e82c81f3a", "date": "Tue, 15 Jul 2025 21:15:47 GMT"} })
10162025-07-15T21:15:47.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10172025-07-15T21:15:47.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10182025-07-15T21:15:47.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10192025-07-15T21:15:47.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10202025-07-15T21:15:47.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10212025-07-15T21:15:47.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10222025-07-15T21:15:47.303ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
10232025-07-15T21:15:47.303ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10242025-07-15T21:15:47.307ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "69cb8add-6b8c-4d2a-8b42-ddb67d33036f", "date": "Tue, 15 Jul 2025 21:15:47 GMT"} })
10252025-07-15T21:15:47.309ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 420.167897ms iteration = 20 status = {}
10262025-07-15T21:15:47.489ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "869e55f8-6495-486f-99de-9a08340b783b", "date": "Tue, 15 Jul 2025 21:15:47 GMT"} })
10272025-07-15T21:15:47.489ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
10282025-07-15T21:15:47.489ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
10292025-07-15T21:15:47.489ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b72a7ae1-a7a5-4cc8-b41c-7f4a886d137a", "content-length": "682", "date": "Tue, 15 Jul 2025 21:15:47 GMT"} })
10302025-07-15T21:15:47.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10312025-07-15T21:15:47.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10322025-07-15T21:15:47.493ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10332025-07-15T21:15:47.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10342025-07-15T21:15:47.496ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10352025-07-15T21:15:47.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10362025-07-15T21:15:47.499ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10372025-07-15T21:15:47.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10382025-07-15T21:15:47.507ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 618.717727ms iteration = 20 status = {}
10392025-07-15T21:15:47.929ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ce40d11d-a13a-445a-b266-ded558a8e2fc roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10402025-07-15T21:15:47.930ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ce40d11d-a13a-445a-b266-ded558a8e2fc resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10412025-07-15T21:15:47.932ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2889 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ce40d11d-a13a-445a-b266-ded558a8e2fc response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10422025-07-15T21:15:48.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d707882a-f16d-4e04-9939-d69efeab611d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10432025-07-15T21:15:48.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d707882a-f16d-4e04-9939-d69efeab611d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10442025-07-15T21:15:48.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1966 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d707882a-f16d-4e04-9939-d69efeab611d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10452025-07-15T21:15:49.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad77d356-65ce-4667-862c-3cdaabbf49e1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10462025-07-15T21:15:49.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad77d356-65ce-4667-862c-3cdaabbf49e1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10472025-07-15T21:15:49.966ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2716 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad77d356-65ce-4667-862c-3cdaabbf49e1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10482025-07-15T21:15:49.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 464 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 9518a966-982a-4736-8fc5-3fd75698c3e9 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
10492025-07-15T21:15:50.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = aa571514-89a1-4242-9ad0-30f3fa46171d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10502025-07-15T21:15:50.981ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = aa571514-89a1-4242-9ad0-30f3fa46171d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10512025-07-15T21:15:50.983ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2324 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = aa571514-89a1-4242-9ad0-30f3fa46171d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10522025-07-15T21:15:51.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 116 reason = Timeout
10532025-07-15T21:15:51.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10542025-07-15T21:15:51.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10552025-07-15T21:15:51.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10562025-07-15T21:15:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10572025-07-15T21:15:51.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10582025-07-15T21:15:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10592025-07-15T21:15:51.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10602025-07-15T21:15:51.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.345924ms iteration = 116 status = {"chicken_switches_updated":false}
10612025-07-15T21:15:51.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8d22df97-fdcc-4c87-b2d8-7b0a9de9f325 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10622025-07-15T21:15:51.996ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8d22df97-fdcc-4c87-b2d8-7b0a9de9f325 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10632025-07-15T21:15:51.998ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1855 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8d22df97-fdcc-4c87-b2d8-7b0a9de9f325 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10642025-07-15T21:15:53.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e63a3a12-3341-4640-85d5-f767e527e2c8 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10652025-07-15T21:15:53.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e63a3a12-3341-4640-85d5-f767e527e2c8 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10662025-07-15T21:15:53.013ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1974 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e63a3a12-3341-4640-85d5-f767e527e2c8 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10672025-07-15T21:15:54.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9bbc76ac-626f-4056-b4cb-8a221ffeffe4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10682025-07-15T21:15:54.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9bbc76ac-626f-4056-b4cb-8a221ffeffe4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10692025-07-15T21:15:54.030ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2624 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9bbc76ac-626f-4056-b4cb-8a221ffeffe4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10702025-07-15T21:15:55.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8ea5b75b-c35a-42a1-866d-f4711bbf7b15 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10712025-07-15T21:15:55.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8ea5b75b-c35a-42a1-866d-f4711bbf7b15 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10722025-07-15T21:15:55.046ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2617 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8ea5b75b-c35a-42a1-866d-f4711bbf7b15 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10732025-07-15T21:15:56.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a7fe0cc5-5218-4540-b06a-97ad9c395be7 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10742025-07-15T21:15:56.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a7fe0cc5-5218-4540-b06a-97ad9c395be7 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10752025-07-15T21:15:56.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2441 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a7fe0cc5-5218-4540-b06a-97ad9c395be7 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
10762025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
10772025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10782025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
10792025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10802025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 117 reason = Timeout
10812025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10822025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10832025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10842025-07-15T21:15:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10852025-07-15T21:15:56.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10862025-07-15T21:15:56.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872025-07-15T21:15:56.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10882025-07-15T21:15:56.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10892025-07-15T21:15:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10902025-07-15T21:15:56.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10912025-07-15T21:15:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10922025-07-15T21:15:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10932025-07-15T21:15:56.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10942025-07-15T21:15:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10952025-07-15T21:15:56.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10962025-07-15T21:15:56.894ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10972025-07-15T21:15:56.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10982025-07-15T21:15:56.894ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.054878ms iteration = 117 status = {"chicken_switches_updated":false}
10992025-07-15T21:15:56.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11002025-07-15T21:15:56.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
11012025-07-15T21:15:56.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11022025-07-15T21:15:56.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11032025-07-15T21:15:56.911ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
11042025-07-15T21:15:56.911ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.087728ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
11052025-07-15T21:15:56.925ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.151932ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11062025-07-15T21:15:57.078ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 07a8ed20-4da5-43db-8fae-4b0a93b36e69 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11072025-07-15T21:15:57.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 07a8ed20-4da5-43db-8fae-4b0a93b36e69 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11082025-07-15T21:15:57.080ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1939 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 07a8ed20-4da5-43db-8fae-4b0a93b36e69 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11092025-07-15T21:15:58.094ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = eb8c0c97-2bea-4838-a480-79cc834f3cd9 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11102025-07-15T21:15:58.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = eb8c0c97-2bea-4838-a480-79cc834f3cd9 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11112025-07-15T21:15:58.096ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2026 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = eb8c0c97-2bea-4838-a480-79cc834f3cd9 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11122025-07-15T21:15:59.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 621c2c4c-e78c-476c-afe7-6774177597de roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11132025-07-15T21:15:59.110ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 621c2c4c-e78c-476c-afe7-6774177597de resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11142025-07-15T21:15:59.112ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1540 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 621c2c4c-e78c-476c-afe7-6774177597de response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11152025-07-15T21:15:59.968ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = c4edb28f-b855-4291-9e3f-9e206a117911 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
11162025-07-15T21:15:59.968ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = c4edb28f-b855-4291-9e3f-9e206a117911 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
11172025-07-15T21:15:59.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3155 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = c4edb28f-b855-4291-9e3f-9e206a117911 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
11182025-07-15T21:15:59.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = fb138f1a-673d-49df-8f20-fe3492f18a05 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
11192025-07-15T21:15:59.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = fb138f1a-673d-49df-8f20-fe3492f18a05 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
11202025-07-15T21:15:59.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = fb138f1a-673d-49df-8f20-fe3492f18a05 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
11212025-07-15T21:15:59.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 454 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = dfdfaf01-5cc3-47ba-b4b3-c939b7c84021 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
11222025-07-15T21:16:00.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 855b92ec-d961-4805-a0c6-88445abd82e2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11232025-07-15T21:16:00.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 855b92ec-d961-4805-a0c6-88445abd82e2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11242025-07-15T21:16:00.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2072 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 855b92ec-d961-4805-a0c6-88445abd82e2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11252025-07-15T21:16:01.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f3a06dba-c195-4977-b678-ab7df4a79b13 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11262025-07-15T21:16:01.144ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f3a06dba-c195-4977-b678-ab7df4a79b13 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11272025-07-15T21:16:01.145ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f3a06dba-c195-4977-b678-ab7df4a79b13 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11282025-07-15T21:16:01.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 118 reason = Timeout
11292025-07-15T21:16:01.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
11302025-07-15T21:16:01.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
11312025-07-15T21:16:01.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11322025-07-15T21:16:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
11332025-07-15T21:16:01.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
11342025-07-15T21:16:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
11352025-07-15T21:16:01.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
11362025-07-15T21:16:01.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.161209ms iteration = 118 status = {"chicken_switches_updated":false}
11372025-07-15T21:16:02.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ae142bac-3543-4ecd-be62-330a923e3d6a roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11382025-07-15T21:16:02.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ae142bac-3543-4ecd-be62-330a923e3d6a resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11392025-07-15T21:16:02.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2350 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ae142bac-3543-4ecd-be62-330a923e3d6a response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11402025-07-15T21:16:03.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 12fa68da-50ea-4d03-9799-92761a135152 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11412025-07-15T21:16:03.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 12fa68da-50ea-4d03-9799-92761a135152 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11422025-07-15T21:16:03.178ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1755 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 12fa68da-50ea-4d03-9799-92761a135152 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11432025-07-15T21:16:04.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e8ca589f-e225-4efe-b246-a56c5e14aa17 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11442025-07-15T21:16:04.193ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e8ca589f-e225-4efe-b246-a56c5e14aa17 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11452025-07-15T21:16:04.195ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2147 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e8ca589f-e225-4efe-b246-a56c5e14aa17 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11462025-07-15T21:16:05.210ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1759fa5a-d1e1-4722-b577-e1bed6fe30a0 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11472025-07-15T21:16:05.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1759fa5a-d1e1-4722-b577-e1bed6fe30a0 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11482025-07-15T21:16:05.212ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2199 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1759fa5a-d1e1-4722-b577-e1bed6fe30a0 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11492025-07-15T21:16:06.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5376ae9-34f0-4796-a441-20ebaa668e8d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11502025-07-15T21:16:06.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5376ae9-34f0-4796-a441-20ebaa668e8d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11512025-07-15T21:16:06.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2562 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5376ae9-34f0-4796-a441-20ebaa668e8d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11522025-07-15T21:16:06.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
11532025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
11542025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11552025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 119 reason = Timeout
11562025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11572025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
11582025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11592025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
11602025-07-15T21:16:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11612025-07-15T21:16:06.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11622025-07-15T21:16:06.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11632025-07-15T21:16:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11642025-07-15T21:16:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
11652025-07-15T21:16:06.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
11662025-07-15T21:16:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
11672025-07-15T21:16:06.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
11682025-07-15T21:16:06.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.359084ms iteration = 119 status = {"chicken_switches_updated":false}
11692025-07-15T21:16:06.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11702025-07-15T21:16:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11712025-07-15T21:16:06.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11722025-07-15T21:16:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11732025-07-15T21:16:06.933ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 47.837766ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11742025-07-15T21:16:06.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11752025-07-15T21:16:06.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11762025-07-15T21:16:06.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772025-07-15T21:16:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
11782025-07-15T21:16:06.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11792025-07-15T21:16:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11802025-07-15T21:16:06.947ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
11812025-07-15T21:16:06.947ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 61.703934ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
11822025-07-15T21:16:07.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 33760e67-fe5d-4982-bd7f-c61c0aaeba37 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11832025-07-15T21:16:07.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 33760e67-fe5d-4982-bd7f-c61c0aaeba37 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11842025-07-15T21:16:07.244ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 33760e67-fe5d-4982-bd7f-c61c0aaeba37 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11852025-07-15T21:16:08.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3d9087c5-1d5c-4830-9308-2c2f6f7235b7 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11862025-07-15T21:16:08.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3d9087c5-1d5c-4830-9308-2c2f6f7235b7 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11872025-07-15T21:16:08.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2572 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3d9087c5-1d5c-4830-9308-2c2f6f7235b7 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11882025-07-15T21:16:09.275ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a3cfeea2-f811-48cb-8812-0391e7715cb7 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11892025-07-15T21:16:09.275ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a3cfeea2-f811-48cb-8812-0391e7715cb7 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11902025-07-15T21:16:09.277ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2040 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a3cfeea2-f811-48cb-8812-0391e7715cb7 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11912025-07-15T21:16:09.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 536 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 5f380fcb-355a-44f2-98b9-8ebb5cc819f0 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
11922025-07-15T21:16:10.290ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e618015-22ff-4e5d-b400-3b47ceb3149b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11932025-07-15T21:16:10.291ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e618015-22ff-4e5d-b400-3b47ceb3149b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11942025-07-15T21:16:10.292ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1970 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e618015-22ff-4e5d-b400-3b47ceb3149b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11952025-07-15T21:16:11.307ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 21b130cd-b976-4037-9cd4-b1aefdec74b3 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11962025-07-15T21:16:11.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 21b130cd-b976-4037-9cd4-b1aefdec74b3 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11972025-07-15T21:16:11.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2110 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 21b130cd-b976-4037-9cd4-b1aefdec74b3 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
11982025-07-15T21:16:11.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 120 reason = Timeout
11992025-07-15T21:16:11.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
12002025-07-15T21:16:11.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
12012025-07-15T21:16:11.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12022025-07-15T21:16:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
12032025-07-15T21:16:11.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
12042025-07-15T21:16:11.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
12052025-07-15T21:16:11.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
12062025-07-15T21:16:11.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.188686ms iteration = 120 status = {"chicken_switches_updated":false}
12072025-07-15T21:16:12.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6a72d08d-d857-4e9c-8307-96635589b8b1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12082025-07-15T21:16:12.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6a72d08d-d857-4e9c-8307-96635589b8b1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12092025-07-15T21:16:12.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2291 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6a72d08d-d857-4e9c-8307-96635589b8b1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12102025-07-15T21:16:13.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c68be27d-c469-424b-9f59-74b114d0b1b5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12112025-07-15T21:16:13.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c68be27d-c469-424b-9f59-74b114d0b1b5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12122025-07-15T21:16:13.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c68be27d-c469-424b-9f59-74b114d0b1b5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12132025-07-15T21:16:14.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 06c02eda-b8d9-4ef5-b301-7a570f28e515 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12142025-07-15T21:16:14.358ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 06c02eda-b8d9-4ef5-b301-7a570f28e515 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12152025-07-15T21:16:14.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2181 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 06c02eda-b8d9-4ef5-b301-7a570f28e515 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12162025-07-15T21:16:15.374ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba35af42-87b7-466b-96ec-e19279c1b1d9 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12172025-07-15T21:16:15.374ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba35af42-87b7-466b-96ec-e19279c1b1d9 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12182025-07-15T21:16:15.375ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2004 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba35af42-87b7-466b-96ec-e19279c1b1d9 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12192025-07-15T21:16:16.390ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 19bdb323-965e-438b-a7ee-669590607ee6 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12202025-07-15T21:16:16.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 19bdb323-965e-438b-a7ee-669590607ee6 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12212025-07-15T21:16:16.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1996 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 19bdb323-965e-438b-a7ee-669590607ee6 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
12222025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
12232025-07-15T21:16:16.815ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225
12242025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.161\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225"}
12252025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
12262025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
12272025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
12282025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
12292025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12302025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12312025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
12322025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
12332025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
12342025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12352025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
12362025-07-15T21:16:16.815ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-07-15 21:06:16.815442572 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
12372025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
12382025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
12392025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
12402025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
12412025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12422025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12432025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
12442025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
12452025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12462025-07-15T21:16:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12472025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
12482025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12492025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12502025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12512025-07-15T21:16:16.816ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4fcf02c2-2876-4bd3-b6eb-3250110a4c40.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 306141772 } })
12522025-07-15T21:16:16.816ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ada38bf-5331-48b0-9340-612e7eca7fae.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1fb7ee6e-8db0-4006-be77-f524ca0c97d3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 306148232 } })
12532025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
12542025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12552025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
12562025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.326234ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
12572025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
12582025-07-15T21:16:16.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.454668ms iteration = 11 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
12592025-07-15T21:16:16.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
12602025-07-15T21:16:16.817ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12612025-07-15T21:16:16.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12622025-07-15T21:16:16.818ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 308526774 } })
12632025-07-15T21:16:16.818ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12642025-07-15T21:16:16.818ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.447439ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
12652025-07-15T21:16:16.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12662025-07-15T21:16:16.818ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
12672025-07-15T21:16:16.819ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-07-15 21:06:16.815442572 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
12682025-07-15T21:16:16.819ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.203578ms iteration = 11 status = {"expiration":"2025-07-15T21:06:16.815442572Z","pruned":[]}
12692025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Timeout
12702025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Timeout
12712025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
12722025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12732025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
12742025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.22\u{b5}s iteration = 14 status = "Disabled"
12752025-07-15T21:16:16.821ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
12762025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
12772025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
12782025-07-15T21:16:16.821ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 311579032 } })
12792025-07-15T21:16:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12802025-07-15T21:16:16.821ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
12812025-07-15T21:16:16.822ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12822025-07-15T21:16:16.822ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12832025-07-15T21:16:16.823ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12842025-07-15T21:16:16.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12852025-07-15T21:16:16.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
12862025-07-15T21:16:16.826ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
12872025-07-15T21:16:16.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
12882025-07-15T21:16:16.827ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12892025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
12902025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12912025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12922025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12932025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12942025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12952025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12962025-07-15T21:16:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12972025-07-15T21:16:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12982025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
12992025-07-15T21:16:16.830ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
13002025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13012025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
13022025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
13032025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
13042025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
13052025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13072025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
13082025-07-15T21:16:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
13092025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
13102025-07-15T21:16:16.834ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
13112025-07-15T21:16:16.834ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
13122025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
13132025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
13142025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
13152025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
13162025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
13172025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
13182025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
13192025-07-15T21:16:16.834ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
13202025-07-15T21:16:16.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
13212025-07-15T21:16:16.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
13222025-07-15T21:16:16.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
13232025-07-15T21:16:16.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
13242025-07-15T21:16:16.835ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
13252025-07-15T21:16:16.835ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13262025-07-15T21:16:16.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13272025-07-15T21:16:16.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13282025-07-15T21:16:16.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13292025-07-15T21:16:16.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13302025-07-15T21:16:16.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13312025-07-15T21:16:16.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13322025-07-15T21:16:16.838ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.253314ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
13332025-07-15T21:16:16.840ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
13342025-07-15T21:16:16.840ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.222515ms iteration = 11 status = {"by_rx":{},"error":null}
13352025-07-15T21:16:16.841ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13362025-07-15T21:16:16.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13372025-07-15T21:16:16.850ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13382025-07-15T21:16:16.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13392025-07-15T21:16:16.851ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
13402025-07-15T21:16:16.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
13412025-07-15T21:16:16.856ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 21.69233ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"162.0.0","globs_reprocessed":{},"no_receivers":[]}
13422025-07-15T21:16:16.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13432025-07-15T21:16:16.885ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
13442025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
13452025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
13462025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13472025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
13482025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13492025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
13502025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
13512025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13522025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13532025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 121 reason = Timeout
13542025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13552025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
13562025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
13572025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
13582025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13592025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13602025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
13612025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13622025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
13632025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13642025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13652025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
13662025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13672025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13682025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
13692025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
13702025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
13712025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13722025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
13732025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
13742025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13752025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
13762025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
13772025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13782025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13792025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
13802025-07-15T21:16:16.886ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
13812025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
13822025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
13832025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13842025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13852025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13862025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13872025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13882025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13892025-07-15T21:16:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13902025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
13912025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13922025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
13932025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
13942025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13952025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13962025-07-15T21:16:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13972025-07-15T21:16:16.887ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 377301140 } })
13982025-07-15T21:16:16.888ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
13992025-07-15T21:16:16.888ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
14002025-07-15T21:16:16.888ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
14012025-07-15T21:16:16.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14022025-07-15T21:16:16.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14032025-07-15T21:16:16.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14042025-07-15T21:16:16.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14052025-07-15T21:16:16.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14062025-07-15T21:16:16.894ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14072025-07-15T21:16:16.895ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14082025-07-15T21:16:16.896ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14092025-07-15T21:16:16.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14102025-07-15T21:16:16.898ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14112025-07-15T21:16:16.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14122025-07-15T21:16:16.900ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
14132025-07-15T21:16:16.900ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14142025-07-15T21:16:16.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
14152025-07-15T21:16:16.900ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14162025-07-15T21:16:16.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14172025-07-15T21:16:16.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14182025-07-15T21:16:16.911ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
14192025-07-15T21:16:16.911ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
14202025-07-15T21:16:16.911ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 90.089828ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
14212025-07-15T21:16:16.919ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14222025-07-15T21:16:16.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
14232025-07-15T21:16:16.923ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14242025-07-15T21:16:16.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14252025-07-15T21:16:16.927ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14262025-07-15T21:16:16.931ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14272025-07-15T21:16:16.931ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14282025-07-15T21:16:16.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14292025-07-15T21:16:16.936ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14302025-07-15T21:16:16.936ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
14312025-07-15T21:16:16.936ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
14322025-07-15T21:16:16.936ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 105.820425ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
14332025-07-15T21:16:16.938ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342025-07-15T21:16:16.939ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
14352025-07-15T21:16:16.939ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14362025-07-15T21:16:16.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14372025-07-15T21:16:16.939ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14382025-07-15T21:16:16.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14392025-07-15T21:16:16.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14402025-07-15T21:16:16.942ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14412025-07-15T21:16:16.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14422025-07-15T21:16:16.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14432025-07-15T21:16:16.944ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14442025-07-15T21:16:16.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14452025-07-15T21:16:16.945ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14462025-07-15T21:16:16.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14472025-07-15T21:16:16.945ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14482025-07-15T21:16:16.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14492025-07-15T21:16:16.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14502025-07-15T21:16:16.947ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14512025-07-15T21:16:16.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14522025-07-15T21:16:16.949ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14532025-07-15T21:16:16.953ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14542025-07-15T21:16:16.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14552025-07-15T21:16:16.953ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
14562025-07-15T21:16:16.953ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14572025-07-15T21:16:16.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14582025-07-15T21:16:16.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
14592025-07-15T21:16:16.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
14602025-07-15T21:16:16.962ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
14612025-07-15T21:16:16.962ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14622025-07-15T21:16:16.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14632025-07-15T21:16:16.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14642025-07-15T21:16:16.963ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14652025-07-15T21:16:16.965ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14662025-07-15T21:16:16.966ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672025-07-15T21:16:16.967ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14682025-07-15T21:16:16.967ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14692025-07-15T21:16:16.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14702025-07-15T21:16:16.967ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712025-07-15T21:16:16.969ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14722025-07-15T21:16:16.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14732025-07-15T21:16:16.971ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742025-07-15T21:16:16.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14752025-07-15T21:16:16.972ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14762025-07-15T21:16:16.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14772025-07-15T21:16:16.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
14782025-07-15T21:16:16.972ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14792025-07-15T21:16:16.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14802025-07-15T21:16:16.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14812025-07-15T21:16:16.973ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14822025-07-15T21:16:16.973ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14832025-07-15T21:16:16.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14842025-07-15T21:16:16.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14852025-07-15T21:16:16.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
14862025-07-15T21:16:16.975ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14872025-07-15T21:16:16.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14882025-07-15T21:16:16.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14892025-07-15T21:16:16.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14902025-07-15T21:16:16.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14912025-07-15T21:16:16.978ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
14922025-07-15T21:16:16.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
14932025-07-15T21:16:16.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
14942025-07-15T21:16:16.994ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14952025-07-15T21:16:16.995ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 108.806641ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
14962025-07-15T21:16:16.998ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14972025-07-15T21:16:16.999ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14982025-07-15T21:16:16.999ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14992025-07-15T21:16:16.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
15002025-07-15T21:16:16.999ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15012025-07-15T21:16:17.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15022025-07-15T21:16:17.003ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15032025-07-15T21:16:17.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15042025-07-15T21:16:17.004ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
15052025-07-15T21:16:17.004ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15062025-07-15T21:16:17.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15072025-07-15T21:16:17.006ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15082025-07-15T21:16:17.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15092025-07-15T21:16:17.007ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15102025-07-15T21:16:17.007ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 120.434192ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
15112025-07-15T21:16:17.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15122025-07-15T21:16:17.008ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 122.085555ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
15132025-07-15T21:16:17.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15142025-07-15T21:16:17.012ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15152025-07-15T21:16:17.012ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15162025-07-15T21:16:17.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15172025-07-15T21:16:17.014ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
15182025-07-15T21:16:17.014ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
15192025-07-15T21:16:17.014ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
15202025-07-15T21:16:17.014ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-07-15 21:06:16.954295 UTC dns_group = internal generation = 1
15212025-07-15T21:16:17.014ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 199.446454ms iteration = 11 status = {"generation":1}
15222025-07-15T21:16:17.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15232025-07-15T21:16:17.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15242025-07-15T21:16:17.015ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15252025-07-15T21:16:17.015ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15262025-07-15T21:16:17.015ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15272025-07-15T21:16:17.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15282025-07-15T21:16:17.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15292025-07-15T21:16:17.021ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
15302025-07-15T21:16:17.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
15312025-07-15T21:16:17.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
15322025-07-15T21:16:17.023ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 136.691382ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15332025-07-15T21:16:17.024ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
15342025-07-15T21:16:17.024ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 138.477469ms iteration = 121 status = {"chicken_switches_updated":false}
15352025-07-15T21:16:17.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15362025-07-15T21:16:17.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15372025-07-15T21:16:17.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15382025-07-15T21:16:17.029ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 519728981 } })
15392025-07-15T21:16:17.030ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
15402025-07-15T21:16:17.032ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
15412025-07-15T21:16:17.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15422025-07-15T21:16:17.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15432025-07-15T21:16:17.034ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
15442025-07-15T21:16:17.034ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 0 version = 2
15452025-07-15T21:16:17.034ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15462025-07-15T21:16:17.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
15472025-07-15T21:16:17.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15482025-07-15T21:16:17.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
15492025-07-15T21:16:17.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15502025-07-15T21:16:17.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15512025-07-15T21:16:17.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15522025-07-15T21:16:17.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15532025-07-15T21:16:17.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15542025-07-15T21:16:17.043ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
15552025-07-15T21:16:17.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
15562025-07-15T21:16:17.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
15572025-07-15T21:16:17.045ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 4 version = 2
15582025-07-15T21:16:17.045ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
15592025-07-15T21:16:17.045ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-07-15 21:06:19.267830 UTC dns_group = external generation = 2
15602025-07-15T21:16:17.045ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 229.857369ms iteration = 11 status = {"generation":2}
15612025-07-15T21:16:17.045ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
15622025-07-15T21:16:17.045ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 210.875249ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
15632025-07-15T21:16:17.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15642025-07-15T21:16:17.051ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
15652025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd126a0b-541a-4fbd-85b2-34f68f4255c3", "content-length": "78", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
15662025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
15672025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
15682025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
15692025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
15702025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
15712025-07-15T21:16:17.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
15722025-07-15T21:16:17.055ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
15732025-07-15T21:16:17.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
15742025-07-15T21:16:17.055ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
15752025-07-15T21:16:17.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
15762025-07-15T21:16:17.057ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
15772025-07-15T21:16:17.058ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
15782025-07-15T21:16:17.058ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
15792025-07-15T21:16:17.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
15802025-07-15T21:16:17.060ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
15812025-07-15T21:16:17.060ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
15822025-07-15T21:16:17.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
15832025-07-15T21:16:17.064ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
15842025-07-15T21:16:17.064ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 243.577413ms iteration = 5 status = {"blueprint_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","inventory_collection_id":"d7e23cb7-4e44-4ca5-bb00-4d4d0c165e78","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
15852025-07-15T21:16:17.067ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
15862025-07-15T21:16:17.067ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 181.747436ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
15872025-07-15T21:16:17.070ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
15882025-07-15T21:16:17.070ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 182.962498ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
15892025-07-15T21:16:17.070ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15902025-07-15T21:16:17.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15912025-07-15T21:16:17.071ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
15922025-07-15T21:16:17.071ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
15932025-07-15T21:16:17.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
15942025-07-15T21:16:17.072ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "91333725-065c-4d7a-8a80-cc9bff316a15", "content-length": "81", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
15952025-07-15T21:16:17.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
15962025-07-15T21:16:17.078ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15972025-07-15T21:16:17.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15982025-07-15T21:16:17.080ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15992025-07-15T21:16:17.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16002025-07-15T21:16:17.081ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16012025-07-15T21:16:17.083ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16022025-07-15T21:16:17.086ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16032025-07-15T21:16:17.086ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16042025-07-15T21:16:17.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
16052025-07-15T21:16:17.087ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16062025-07-15T21:16:17.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16072025-07-15T21:16:17.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16082025-07-15T21:16:17.091ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 204.294159ms iteration = 21 status = 0
16092025-07-15T21:16:17.094ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 207.336738ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16102025-07-15T21:16:17.096ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16112025-07-15T21:16:17.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16122025-07-15T21:16:17.097ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 211.299419ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16132025-07-15T21:16:17.098ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16142025-07-15T21:16:17.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16152025-07-15T21:16:17.104ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16162025-07-15T21:16:17.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16172025-07-15T21:16:17.109ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16182025-07-15T21:16:17.109ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16192025-07-15T21:16:17.109ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16202025-07-15T21:16:17.109ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16212025-07-15T21:16:17.109ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16222025-07-15T21:16:17.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16232025-07-15T21:16:17.110ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
16242025-07-15T21:16:17.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16252025-07-15T21:16:17.110ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 224.048689ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16262025-07-15T21:16:17.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16272025-07-15T21:16:17.113ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 227.423067ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16282025-07-15T21:16:17.113ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16292025-07-15T21:16:17.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16302025-07-15T21:16:17.116ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16312025-07-15T21:16:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16322025-07-15T21:16:17.118ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(7096f6c9-b78e-4222-91a9-684eb6bf4b77, 0), (e72fb205-ca15-456e-8e76-cccbc1b2b241, 0), (d7e23cb7-4e44-4ca5-bb00-4d4d0c165e78, 0)]
16332025-07-15T21:16:17.118ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1584, tv_nsec: 608759481 } })
16342025-07-15T21:16:17.119ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16352025-07-15T21:16:17.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16362025-07-15T21:16:17.122ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16372025-07-15T21:16:17.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16382025-07-15T21:16:17.124ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16392025-07-15T21:16:17.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16402025-07-15T21:16:17.127ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16412025-07-15T21:16:17.131ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
16422025-07-15T21:16:17.134ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
16432025-07-15T21:16:17.134ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
16442025-07-15T21:16:17.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
16452025-07-15T21:16:17.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16462025-07-15T21:16:17.134ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16472025-07-15T21:16:17.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16482025-07-15T21:16:17.138ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16492025-07-15T21:16:17.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16502025-07-15T21:16:17.141ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
16512025-07-15T21:16:17.142ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
16522025-07-15T21:16:17.142ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "5770b3fe-8ff8-42e4-92b7-b1da00ba8810", "content-length": "81", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
16532025-07-15T21:16:17.142ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16542025-07-15T21:16:17.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16552025-07-15T21:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16562025-07-15T21:16:17.145ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
16572025-07-15T21:16:17.145ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
16582025-07-15T21:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16592025-07-15T21:16:17.145ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "561b07ab-d321-4e04-a96b-77ea1f084d05", "content-length": "81", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
16602025-07-15T21:16:17.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16612025-07-15T21:16:17.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16622025-07-15T21:16:17.147ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 261.217419ms iteration = 21 status = {"errors":[],"sps":[]}
16632025-07-15T21:16:17.150ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16642025-07-15T21:16:17.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16652025-07-15T21:16:17.152ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16662025-07-15T21:16:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16672025-07-15T21:16:17.155ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16682025-07-15T21:16:17.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16692025-07-15T21:16:17.157ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16702025-07-15T21:16:17.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16712025-07-15T21:16:17.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16722025-07-15T21:16:17.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16732025-07-15T21:16:17.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16742025-07-15T21:16:17.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16752025-07-15T21:16:17.163ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-07-15T21:06:12.596852Z, time_modified: 2025-07-15T21:06:12.596852Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }) }
16762025-07-15T21:16:17.163ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 347.763426ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]}},"default_endpoint":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
16772025-07-15T21:16:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16782025-07-15T21:16:17.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
16792025-07-15T21:16:17.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16802025-07-15T21:16:17.164ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16812025-07-15T21:16:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16822025-07-15T21:16:17.166ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16832025-07-15T21:16:17.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16842025-07-15T21:16:17.167ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16852025-07-15T21:16:17.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16862025-07-15T21:16:17.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16872025-07-15T21:16:17.168ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16882025-07-15T21:16:17.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16892025-07-15T21:16:17.170ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16902025-07-15T21:16:17.170ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
16912025-07-15T21:16:17.170ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16922025-07-15T21:16:17.171ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "58c81736-c1b7-4989-9c29-ae38344544bb", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
16932025-07-15T21:16:17.171ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16942025-07-15T21:16:17.173ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16952025-07-15T21:16:17.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
16962025-07-15T21:16:17.176ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16972025-07-15T21:16:17.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16982025-07-15T21:16:17.181ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16992025-07-15T21:16:17.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17002025-07-15T21:16:17.184ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17012025-07-15T21:16:17.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17022025-07-15T21:16:17.186ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17032025-07-15T21:16:17.186ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
17042025-07-15T21:16:17.186ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 300.832224ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
17052025-07-15T21:16:17.187ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17062025-07-15T21:16:17.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17072025-07-15T21:16:17.190ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17082025-07-15T21:16:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17092025-07-15T21:16:17.193ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17102025-07-15T21:16:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17112025-07-15T21:16:17.195ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17122025-07-15T21:16:17.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17132025-07-15T21:16:17.197ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17142025-07-15T21:16:17.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17152025-07-15T21:16:17.199ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17162025-07-15T21:16:17.205ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
17172025-07-15T21:16:17.205ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17182025-07-15T21:16:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17192025-07-15T21:16:17.207ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17202025-07-15T21:16:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17212025-07-15T21:16:17.209ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17222025-07-15T21:16:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17232025-07-15T21:16:17.211ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17242025-07-15T21:16:17.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
17252025-07-15T21:16:17.216ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17262025-07-15T21:16:17.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17272025-07-15T21:16:17.222ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
17282025-07-15T21:16:17.222ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17292025-07-15T21:16:17.223ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03e5d1b7-fba8-4703-9586-2b867887aae3", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17302025-07-15T21:16:17.223ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17312025-07-15T21:16:17.224ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 338.62805ms iteration = 23 status = {"physical_disks_added":0}
17322025-07-15T21:16:17.225ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 72dc1995-096d-45df-890a-d52ba7e736b5, kind: Service { id: 5a78b25c-4f60-42ae-bd1b-943e9cd5c1f1 }, name: Name("nexus-5a78b25c-4f60-42ae-bd1b-943e9cd5c1f1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 78])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9504f0e5-30d9-4326-855d-908673f5704e, kind: Service { id: 6e23f75f-139f-4fef-bc78-c2a2188cf2e9 }, name: Name("nexus-6e23f75f-139f-4fef-bc78-c2a2188cf2e9"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 244])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be0f14fa-e706-4486-a825-adc9fbbe0b34, kind: Service { id: 23b4c93e-80af-4ddc-945f-e182e8dce33c }, name: Name("nexus-23b4c93e-80af-4ddc-945f-e182e8dce33c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 197, 23])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 41a68e84-391c-48e3-9e33-25ec9126f993, kind: Service { id: 77d2eb9f-f483-4c87-9935-5142541c93df }, name: Name("external-dns-77d2eb9f-f483-4c87-9935-5142541c93df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 135, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 46757b9b-3666-4542-8d0b-1bf567389c05, kind: Service { id: 4fcf02c2-2876-4bd3-b6eb-3250110a4c40 }, name: Name("external-dns-4fcf02c2-2876-4bd3-b6eb-3250110a4c40"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
17332025-07-15T21:16:17.225ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17342025-07-15T21:16:17.225ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17352025-07-15T21:16:17.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17362025-07-15T21:16:17.227ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17372025-07-15T21:16:17.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
17382025-07-15T21:16:17.231ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17392025-07-15T21:16:17.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17402025-07-15T21:16:17.233ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
17412025-07-15T21:16:17.233ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a68e84-391c-48e3-9e33-25ec9126f993, kind: Service { id: 77d2eb9f-f483-4c87-9935-5142541c93df }, name: Name("external-dns-77d2eb9f-f483-4c87-9935-5142541c93df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 135, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 46757b9b-3666-4542-8d0b-1bf567389c05, kind: Service { id: 4fcf02c2-2876-4bd3-b6eb-3250110a4c40 }, name: Name("external-dns-4fcf02c2-2876-4bd3-b6eb-3250110a4c40"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 72dc1995-096d-45df-890a-d52ba7e736b5, kind: Service { id: 5a78b25c-4f60-42ae-bd1b-943e9cd5c1f1 }, name: Name("nexus-5a78b25c-4f60-42ae-bd1b-943e9cd5c1f1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 78])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9504f0e5-30d9-4326-855d-908673f5704e, kind: Service { id: 6e23f75f-139f-4fef-bc78-c2a2188cf2e9 }, name: Name("nexus-6e23f75f-139f-4fef-bc78-c2a2188cf2e9"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 244])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be0f14fa-e706-4486-a825-adc9fbbe0b34, kind: Service { id: 23b4c93e-80af-4ddc-945f-e182e8dce33c }, name: Name("nexus-23b4c93e-80af-4ddc-945f-e182e8dce33c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 197, 23])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 41a68e84-391c-48e3-9e33-25ec9126f993, kind: Service { id: 77d2eb9f-f483-4c87-9935-5142541c93df }, name: Name("external-dns-77d2eb9f-f483-4c87-9935-5142541c93df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 135, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 46757b9b-3666-4542-8d0b-1bf567389c05, kind: Service { id: 4fcf02c2-2876-4bd3-b6eb-3250110a4c40 }, name: Name("external-dns-4fcf02c2-2876-4bd3-b6eb-3250110a4c40"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 72dc1995-096d-45df-890a-d52ba7e736b5, kind: Service { id: 5a78b25c-4f60-42ae-bd1b-943e9cd5c1f1 }, name: Name("nexus-5a78b25c-4f60-42ae-bd1b-943e9cd5c1f1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 78])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9504f0e5-30d9-4326-855d-908673f5704e, kind: Service { id: 6e23f75f-139f-4fef-bc78-c2a2188cf2e9 }, name: Name("nexus-6e23f75f-139f-4fef-bc78-c2a2188cf2e9"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 244])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be0f14fa-e706-4486-a825-adc9fbbe0b34, kind: Service { id: 23b4c93e-80af-4ddc-945f-e182e8dce33c }, name: Name("nexus-23b4c93e-80af-4ddc-945f-e182e8dce33c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 197, 23])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
17422025-07-15T21:16:17.233ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
17432025-07-15T21:16:17.235ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
17442025-07-15T21:16:17.295ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
17452025-07-15T21:16:17.358ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
17462025-07-15T21:16:17.358ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 528.130846ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
17472025-07-15T21:16:17.386ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 12003c51-8cc1-4676-bb02-79c3a895e0eb, time_created: 2025-07-15T21:06:11.686655Z, time_modified: 2025-07-15T21:06:11.686655Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
17482025-07-15T21:16:17.386ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
17492025-07-15T21:16:17.406ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 02127b2b-20e1-4bcf-ac90-a04c6e815698 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
17502025-07-15T21:16:17.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 02127b2b-20e1-4bcf-ac90-a04c6e815698 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
17512025-07-15T21:16:17.408ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
17522025-07-15T21:16:17.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
17532025-07-15T21:16:17.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 5868 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 02127b2b-20e1-4bcf-ac90-a04c6e815698 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
17542025-07-15T21:16:17.421ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17552025-07-15T21:16:17.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
17562025-07-15T21:16:17.425ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
17572025-07-15T21:16:17.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
17582025-07-15T21:16:17.428ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
17592025-07-15T21:16:17.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
17602025-07-15T21:16:17.430ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17612025-07-15T21:16:17.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
17622025-07-15T21:16:17.434ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
17632025-07-15T21:16:17.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
17642025-07-15T21:16:17.437ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
17652025-07-15T21:16:17.438ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "01db0f93-d2de-4a47-ad76-9e973c5d9f45", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17662025-07-15T21:16:17.438ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
17672025-07-15T21:16:17.438ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0017337-4fb5-41c5-813a-4ccc9ecd3465", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17682025-07-15T21:16:17.438ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:672 sled = http://[fd00:1122:3344:101::1]:12345
17692025-07-15T21:16:17.440ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 605.991188ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"12003c51-8cc1-4676-bb02-79c3a895e0eb","time":"2025-07-15T21:06:17.620453912Z"},{"operation":"list","target_sled":"12003c51-8cc1-4676-bb02-79c3a895e0eb","time":"2025-07-15T21:06:17.634019432Z"},{"generation":1,"operation":"put_config","target_sled":"12003c51-8cc1-4676-bb02-79c3a895e0eb","time":"2025-07-15T21:11:17.721570443Z"},{"operation":"list","target_sled":"12003c51-8cc1-4676-bb02-79c3a895e0eb","time":"2025-07-15T21:11:17.722037625Z"},{"generation":1,"operation":"put_config","target_sled":"12003c51-8cc1-4676-bb02-79c3a895e0eb","time":"2025-07-15T21:16:17.438145792Z"},{"operation":"list","target_sled":"12003c51-8cc1-4676-bb02-79c3a895e0eb","time":"2025-07-15T21:16:17.438469390Z"}]}
17702025-07-15T21:16:17.450ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
17712025-07-15T21:16:17.450ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17722025-07-15T21:16:17.451ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e539332-5e49-4595-84ef-8b5f2516da8b", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17732025-07-15T21:16:17.451ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17742025-07-15T21:16:17.454ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
17752025-07-15T21:16:17.609ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17762025-07-15T21:16:17.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17772025-07-15T21:16:17.613ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 798.109213ms iteration = 21 status = null
17782025-07-15T21:16:17.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2b2c307a-d05f-4bde-bb66-87f261724bfa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3f9aa3f7-0814-48a6-9b4e-dde33993966b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("842f5c09-b97e-40b2-8df8-5ccc06c66cdb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("8fb89e04-54a6-4c8f-9e69-4b407b46ea88.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("aa37ee39-c7cf-44a3-8a10-00ca23518ac5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("aa37ee39-c7cf-44a3-8a10-00ca23518ac5.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1585, tv_nsec: 114157271 } })
17792025-07-15T21:16:17.625ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
17802025-07-15T21:16:17.625ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
17812025-07-15T21:16:17.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
17822025-07-15T21:16:17.625ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d57cac6b-d9df-4f0a-b652-d279f755c798", "content-length": "476", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17832025-07-15T21:16:17.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
17842025-07-15T21:16:17.626ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc20d26e-0af8-45bf-a4d9-954b9bdcf832", "content-length": "734", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17852025-07-15T21:16:17.626ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
17862025-07-15T21:16:17.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b6567f1-685d-46e7-b455-580670a65bda", "content-length": "111", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17872025-07-15T21:16:17.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
17882025-07-15T21:16:17.627ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 812.060083ms iteration = 21 status = {}
17892025-07-15T21:16:17.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "867ca7d7-e7f6-45e6-827b-0a49589f2b7a", "content-length": "111", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17902025-07-15T21:16:17.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
17912025-07-15T21:16:17.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb2fcaad-b034-4e77-9ca6-dd86d869be63", "content-length": "170", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17922025-07-15T21:16:17.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
17932025-07-15T21:16:17.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a055d1d-7775-492d-a7ce-3af8fa89e931", "content-length": "170", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17942025-07-15T21:16:17.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
17952025-07-15T21:16:17.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee12323f-c588-4b7f-af22-6f544d391f23", "content-length": "179", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17962025-07-15T21:16:17.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
17972025-07-15T21:16:17.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c69c7dbe-4363-417c-b6d8-3d7bbeb73878", "content-length": "179", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
17982025-07-15T21:16:17.631ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
17992025-07-15T21:16:17.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6af9ed38-0b6a-4025-9031-5d07c759f357", "content-length": "702", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18002025-07-15T21:16:17.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
18012025-07-15T21:16:17.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8964c856-15dc-4d5f-85ad-e8ff87a64b14", "content-length": "727", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18022025-07-15T21:16:17.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
18032025-07-15T21:16:17.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c68b259-7b33-4032-9eaa-fe08380a74e0", "content-length": "729", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18042025-07-15T21:16:17.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
18052025-07-15T21:16:17.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc89c04a-4044-4796-b92f-8aa998700396", "content-length": "728", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18062025-07-15T21:16:17.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
18072025-07-15T21:16:17.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fa7eadb-5f42-41a8-a369-63e579522b50", "content-length": "722", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18082025-07-15T21:16:17.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
18092025-07-15T21:16:17.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef59e6e3-097a-4037-9e21-b652b22e8d57", "content-length": "109", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18102025-07-15T21:16:17.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
18112025-07-15T21:16:17.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e11fced0-6879-4c8e-aab5-121355c9ed89", "content-length": "109", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18122025-07-15T21:16:17.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
18132025-07-15T21:16:17.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "609bc08c-3ff1-4869-a3bb-80b63fe4469f", "content-length": "169", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18142025-07-15T21:16:17.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
18152025-07-15T21:16:17.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "716a37bf-3cf0-404a-8e4d-b382330916eb", "content-length": "169", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18162025-07-15T21:16:17.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
18172025-07-15T21:16:17.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c72c3523-badb-493b-a0ad-077cea576ed5", "content-length": "178", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18182025-07-15T21:16:17.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
18192025-07-15T21:16:17.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "144a17ab-19ca-4727-9347-1ccc3caf2c60", "content-length": "178", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18202025-07-15T21:16:17.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
18212025-07-15T21:16:17.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a76a957-5414-4ebd-a3ab-e313ade6abd7", "content-length": "702", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18222025-07-15T21:16:17.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
18232025-07-15T21:16:17.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74bc5d75-1b9e-44f4-bc1a-6b2941ace48c", "content-length": "727", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18242025-07-15T21:16:17.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
18252025-07-15T21:16:17.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4046d658-c16d-4061-a364-ad2a5feef270", "content-length": "729", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18262025-07-15T21:16:17.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
18272025-07-15T21:16:17.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a47ab4d-dde0-4323-885a-ecc45770ef83", "content-length": "728", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18282025-07-15T21:16:17.637ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18292025-07-15T21:16:17.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18302025-07-15T21:16:17.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18312025-07-15T21:16:17.642ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
18322025-07-15T21:16:17.642ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18332025-07-15T21:16:17.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18342025-07-15T21:16:17.645ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18352025-07-15T21:16:17.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
18362025-07-15T21:16:17.649ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18372025-07-15T21:16:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18382025-07-15T21:16:17.662ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
18392025-07-15T21:16:17.662ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
18402025-07-15T21:16:17.663ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2db5c95-fdd6-48a1-accf-0d9baae73299", "content-length": "372", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18412025-07-15T21:16:17.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18422025-07-15T21:16:17.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18432025-07-15T21:16:17.666ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
18442025-07-15T21:16:17.667ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "08294215-8904-4fcc-b5d1-f9c91aa21e15", "content-length": "691", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18452025-07-15T21:16:17.667ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
18462025-07-15T21:16:17.667ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
18472025-07-15T21:16:17.667ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
18482025-07-15T21:16:17.669ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 783.739971ms iteration = 21 status = {}
18492025-07-15T21:16:17.687ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
18502025-07-15T21:16:17.687ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
18512025-07-15T21:16:17.688ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f31633af-25c0-4442-a43d-76d4b88f7d2c", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18522025-07-15T21:16:17.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18532025-07-15T21:16:17.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18542025-07-15T21:16:17.691ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
18552025-07-15T21:16:17.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18562025-07-15T21:16:17.692ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f3ab7dfb-91b7-4aa6-a67b-04ac3eb5072f", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18572025-07-15T21:16:17.692ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18582025-07-15T21:16:17.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18592025-07-15T21:16:17.694ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "fff826ef-f82c-456e-b3b2-5b4ade557112", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18602025-07-15T21:16:17.694ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18612025-07-15T21:16:17.696ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18622025-07-15T21:16:17.696ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 880.884681ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
18632025-07-15T21:16:17.696ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18642025-07-15T21:16:17.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18652025-07-15T21:16:17.697ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18662025-07-15T21:16:17.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18672025-07-15T21:16:17.700ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18682025-07-15T21:16:17.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18692025-07-15T21:16:17.701ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
18702025-07-15T21:16:17.701ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
18712025-07-15T21:16:17.701ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "aa9742f7-0a66-491d-8062-f225afed0576", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18722025-07-15T21:16:17.701ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
18732025-07-15T21:16:17.702ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "caf68914-33b8-49ae-9f68-4c08796a1169", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18742025-07-15T21:16:17.702ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18752025-07-15T21:16:17.702ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18762025-07-15T21:16:17.703ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 816.9598ms iteration = 21 status = {}
18772025-07-15T21:16:17.705ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "29511b01-0054-4ed4-bec1-c9a620838ee1", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18782025-07-15T21:16:17.705ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18792025-07-15T21:16:17.707ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
18802025-07-15T21:16:17.707ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18812025-07-15T21:16:17.707ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "6ff66f37-ea7e-4933-95ae-9d3c53d43047", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18822025-07-15T21:16:17.707ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18832025-07-15T21:16:17.707ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "f187c388-a3fe-4038-9e35-4af701b3cbb7", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18842025-07-15T21:16:17.709ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
18852025-07-15T21:16:17.709ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 879.624619ms
18862025-07-15T21:16:17.709ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 879.638299ms iteration = 3 status = {}
18872025-07-15T21:16:17.710ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18882025-07-15T21:16:17.710ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 895.058548ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
18892025-07-15T21:16:17.744ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
18902025-07-15T21:16:17.745ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
18912025-07-15T21:16:17.804ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e9e100c-71c7-4c1c-a344-461158626359", "content-length": "88896", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
18922025-07-15T21:16:17.807ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
18932025-07-15T21:16:17.807ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
18942025-07-15T21:16:17.807ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
18952025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
18962025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
18972025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
18982025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
18992025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
19002025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a137339b-75c6-4ea2-a657-caa13baf534e", "content-length": "64", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19012025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
19022025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "09aab246-884e-4e93-ba31-446fb986616c", "content-length": "64", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19032025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7449da28-6b5f-4513-a8b3-c87a4c7581c1", "content-length": "64", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19042025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc34a740-e188-4363-a956-91f7e8391fe4", "content-length": "64", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19052025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc5186bf-82cc-42a8-9f83-ac7c133e9fc0", "content-length": "64", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19062025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
19072025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
19082025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
19092025-07-15T21:16:17.807ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
19102025-07-15T21:16:17.825ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19112025-07-15T21:16:17.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19122025-07-15T21:16:17.828ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
19132025-07-15T21:16:17.828ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19142025-07-15T21:16:17.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19152025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
19162025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
19172025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
19182025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
19192025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19202025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19212025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19222025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19232025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19242025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19252025-07-15T21:16:17.842ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
19262025-07-15T21:16:17.842ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19272025-07-15T21:16:17.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19282025-07-15T21:16:17.845ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
19292025-07-15T21:16:17.845ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19302025-07-15T21:16:17.863ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "98d1f1ef-e5f8-4447-be07-9679cbad5b82", "content-length": "393598", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19312025-07-15T21:16:17.864ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "570aa848-52e2-4154-a726-d05d1786cc5f", "content-length": "430153", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19322025-07-15T21:16:17.871ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2377d038-d3ce-48ff-9468-fb68ee448120", "content-length": "398781", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19332025-07-15T21:16:17.874ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a78ea4a2-6cc8-4b79-aabe-cf47fd107009", "content-length": "390831", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19342025-07-15T21:16:17.882ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1524ca34-17b9-4b0f-b675-f497b1296c8c", "content-length": "388373", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19352025-07-15T21:16:17.884ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
19362025-07-15T21:16:17.884ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
19372025-07-15T21:16:17.884ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
19382025-07-15T21:16:17.884ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
19392025-07-15T21:16:17.884ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
19402025-07-15T21:16:17.884ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
19412025-07-15T21:16:17.886ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19422025-07-15T21:16:17.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19432025-07-15T21:16:17.889ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19442025-07-15T21:16:17.891ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
19452025-07-15T21:16:17.891ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19462025-07-15T21:16:17.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19472025-07-15T21:16:18.115ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 6d60d48f-ac8d-4566-89bc-44a696bb3704 file = nexus/db-queries/src/db/datastore/inventory.rs:1442
19482025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 6d60d48f-ac8d-4566-89bc-44a696bb3704 time_started = 2025-07-15 21:16:17.625202 UTC
19492025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
19502025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Dependency
19512025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19522025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
19532025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
19542025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 15.39\u{b5}s iteration = 15 status = "Disabled"
19552025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
19562025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "be25dcc1-842c-47df-a919-1ac2b201f041", "date": "Tue, 15 Jul 2025 21:16:17 GMT"} })
19572025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.298519912s iteration = 3 status = {"collection_id":"6d60d48f-ac8d-4566-89bc-44a696bb3704","time_done":"2025-07-15 21:16:17.884829 UTC","time_started":"2025-07-15 21:16:17.625202 UTC"}
19582025-07-15T21:16:18.115ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
19592025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19602025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
19612025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19622025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "936752ed-7fcc-43ce-ae6d-9f13d0c5d987", "content-length": "682", "date": "Tue, 15 Jul 2025 21:16:18 GMT"} })
19632025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19642025-07-15T21:16:18.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19652025-07-15T21:16:18.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19662025-07-15T21:16:18.118ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19672025-07-15T21:16:18.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19682025-07-15T21:16:18.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
19692025-07-15T21:16:18.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
19702025-07-15T21:16:18.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19712025-07-15T21:16:18.122ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19722025-07-15T21:16:18.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19732025-07-15T21:16:18.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
19742025-07-15T21:16:18.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19752025-07-15T21:16:18.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
19762025-07-15T21:16:18.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
19772025-07-15T21:16:18.127ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19782025-07-15T21:16:18.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19792025-07-15T21:16:18.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19802025-07-15T21:16:18.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19812025-07-15T21:16:18.129ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822025-07-15T21:16:18.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19832025-07-15T21:16:18.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19842025-07-15T21:16:18.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19852025-07-15T21:16:18.131ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19862025-07-15T21:16:18.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19872025-07-15T21:16:18.140ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.254546277s iteration = 21 status = {}
19882025-07-15T21:16:18.187ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 71.947859ms iteration = 24 status = {"physical_disks_added":0}
19892025-07-15T21:16:18.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
19902025-07-15T21:16:18.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
19912025-07-15T21:16:18.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19922025-07-15T21:16:18.207ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
19932025-07-15T21:16:18.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
19942025-07-15T21:16:18.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
19952025-07-15T21:16:18.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19962025-07-15T21:16:18.211ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
19972025-07-15T21:16:18.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
19982025-07-15T21:16:18.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
19992025-07-15T21:16:18.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20002025-07-15T21:16:18.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20012025-07-15T21:16:18.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20022025-07-15T21:16:18.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20032025-07-15T21:16:18.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20042025-07-15T21:16:18.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20052025-07-15T21:16:18.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20062025-07-15T21:16:18.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20072025-07-15T21:16:18.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20082025-07-15T21:16:18.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20092025-07-15T21:16:18.225ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
20102025-07-15T21:16:18.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 110.570297ms iteration = 6 status = {"blueprint_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","inventory_collection_id":"6d60d48f-ac8d-4566-89bc-44a696bb3704","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
20112025-07-15T21:16:18.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7f361cf6-415d-4c2c-b80a-5b8fe7186ec1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20122025-07-15T21:16:18.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7f361cf6-415d-4c2c-b80a-5b8fe7186ec1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20132025-07-15T21:16:18.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2759 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7f361cf6-415d-4c2c-b80a-5b8fe7186ec1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20142025-07-15T21:16:19.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 690d6d95-d151-4492-9518-9c89bbc32f69 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20152025-07-15T21:16:19.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 690d6d95-d151-4492-9518-9c89bbc32f69 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20162025-07-15T21:16:19.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 690d6d95-d151-4492-9518-9c89bbc32f69 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20172025-07-15T21:16:19.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 519 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 978b7486-24cc-4db5-9e94-6233587f68af response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
20182025-07-15T21:16:20.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 47c4f9f5-1a1d-4df8-86d8-1706ec5d4887 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20192025-07-15T21:16:20.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 47c4f9f5-1a1d-4df8-86d8-1706ec5d4887 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20202025-07-15T21:16:20.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2744 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 47c4f9f5-1a1d-4df8-86d8-1706ec5d4887 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20212025-07-15T21:16:21.493ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c2fa7a95-1b27-4059-baf4-747f8df60646 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20222025-07-15T21:16:21.493ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c2fa7a95-1b27-4059-baf4-747f8df60646 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20232025-07-15T21:16:21.495ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2651 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c2fa7a95-1b27-4059-baf4-747f8df60646 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20242025-07-15T21:16:21.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 122 reason = Timeout
20252025-07-15T21:16:21.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
20262025-07-15T21:16:21.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
20272025-07-15T21:16:21.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20282025-07-15T21:16:21.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
20292025-07-15T21:16:21.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
20302025-07-15T21:16:21.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
20312025-07-15T21:16:21.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
20322025-07-15T21:16:21.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.852766ms iteration = 122 status = {"chicken_switches_updated":false}
20332025-07-15T21:16:22.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fb16440b-1505-493a-92f4-1a2d2bbef784 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20342025-07-15T21:16:22.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fb16440b-1505-493a-92f4-1a2d2bbef784 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20352025-07-15T21:16:22.511ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2465 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fb16440b-1505-493a-92f4-1a2d2bbef784 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20362025-07-15T21:16:23.525ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2bb72e57-8227-4b11-9ca7-cfebf811118e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20372025-07-15T21:16:23.525ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2bb72e57-8227-4b11-9ca7-cfebf811118e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20382025-07-15T21:16:23.527ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2428 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2bb72e57-8227-4b11-9ca7-cfebf811118e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20392025-07-15T21:16:24.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b1403ba-44f0-457c-b997-222815083dae roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20402025-07-15T21:16:24.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b1403ba-44f0-457c-b997-222815083dae resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20412025-07-15T21:16:24.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2118 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b1403ba-44f0-457c-b997-222815083dae response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20422025-07-15T21:16:25.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c0e0b88c-bb44-40a8-834d-be1b1a60cb0b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20432025-07-15T21:16:25.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c0e0b88c-bb44-40a8-834d-be1b1a60cb0b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20442025-07-15T21:16:25.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1882 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c0e0b88c-bb44-40a8-834d-be1b1a60cb0b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20452025-07-15T21:16:26.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 759f8490-42a9-4daf-851a-d7ebb6f10e14 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20462025-07-15T21:16:26.573ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 759f8490-42a9-4daf-851a-d7ebb6f10e14 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20472025-07-15T21:16:26.575ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2242 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 759f8490-42a9-4daf-851a-d7ebb6f10e14 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20482025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
20492025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
20502025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20512025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20522025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 123 reason = Timeout
20532025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
20542025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20552025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20562025-07-15T21:16:26.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
20572025-07-15T21:16:26.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20582025-07-15T21:16:26.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20592025-07-15T21:16:26.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20602025-07-15T21:16:26.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20612025-07-15T21:16:26.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20622025-07-15T21:16:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
20632025-07-15T21:16:26.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
20642025-07-15T21:16:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
20652025-07-15T21:16:26.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20662025-07-15T21:16:26.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20672025-07-15T21:16:26.894ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
20682025-07-15T21:16:26.894ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.701399ms iteration = 123 status = {"chicken_switches_updated":false}
20692025-07-15T21:16:26.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20702025-07-15T21:16:26.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
20712025-07-15T21:16:26.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20722025-07-15T21:16:26.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20732025-07-15T21:16:26.910ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
20742025-07-15T21:16:26.910ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.475792ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
20752025-07-15T21:16:26.923ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20762025-07-15T21:16:26.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20772025-07-15T21:16:26.924ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.540336ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20782025-07-15T21:16:27.587ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cc1b726e-42c5-457e-b133-0cc898c92528 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20792025-07-15T21:16:27.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cc1b726e-42c5-457e-b133-0cc898c92528 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20802025-07-15T21:16:27.589ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1969 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cc1b726e-42c5-457e-b133-0cc898c92528 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20812025-07-15T21:16:28.603ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e8d33668-a3a9-4ef4-ae3f-ea7238a09ed5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20822025-07-15T21:16:28.603ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e8d33668-a3a9-4ef4-ae3f-ea7238a09ed5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20832025-07-15T21:16:28.605ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2444 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e8d33668-a3a9-4ef4-ae3f-ea7238a09ed5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20842025-07-15T21:16:29.619ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fb8bde32-7953-4bfe-ab8a-1535925cb552 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20852025-07-15T21:16:29.619ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fb8bde32-7953-4bfe-ab8a-1535925cb552 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20862025-07-15T21:16:29.621ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2554 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fb8bde32-7953-4bfe-ab8a-1535925cb552 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20872025-07-15T21:16:29.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 1a5a3821-65e7-4bc2-8b49-effb531a7b49 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
20882025-07-15T21:16:29.968ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 1a5a3821-65e7-4bc2-8b49-effb531a7b49 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
20892025-07-15T21:16:29.970ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3082 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 1a5a3821-65e7-4bc2-8b49-effb531a7b49 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
20902025-07-15T21:16:29.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 77168b66-2484-471f-99fe-2e1cc9bee2a4 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
20912025-07-15T21:16:29.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 77168b66-2484-471f-99fe-2e1cc9bee2a4 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
20922025-07-15T21:16:29.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2302 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42847 req_id = 77168b66-2484-471f-99fe-2e1cc9bee2a4 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
20932025-07-15T21:16:29.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 434 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = d5740f5f-f0fd-47d3-a923-0e6076559023 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
20942025-07-15T21:16:30.636ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fdde123b-afac-4964-8b93-917057d56938 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20952025-07-15T21:16:30.636ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fdde123b-afac-4964-8b93-917057d56938 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20962025-07-15T21:16:30.638ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1840 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fdde123b-afac-4964-8b93-917057d56938 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20972025-07-15T21:16:31.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 69e82825-2437-42bd-b66e-41783b71df51 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20982025-07-15T21:16:31.655ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 69e82825-2437-42bd-b66e-41783b71df51 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
20992025-07-15T21:16:31.656ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2332 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 69e82825-2437-42bd-b66e-41783b71df51 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21002025-07-15T21:16:31.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 124 reason = Timeout
21012025-07-15T21:16:31.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
21022025-07-15T21:16:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
21032025-07-15T21:16:31.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21042025-07-15T21:16:31.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
21052025-07-15T21:16:31.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
21062025-07-15T21:16:31.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
21072025-07-15T21:16:31.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
21082025-07-15T21:16:31.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.19238ms iteration = 124 status = {"chicken_switches_updated":false}
21092025-07-15T21:16:32.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8f65d34d-ef0b-47d2-b80b-e3181a446186 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21102025-07-15T21:16:32.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8f65d34d-ef0b-47d2-b80b-e3181a446186 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21112025-07-15T21:16:32.672ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8f65d34d-ef0b-47d2-b80b-e3181a446186 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21122025-07-15T21:16:33.687ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0bbdeef7-f265-490b-b4b1-729384ad0bc5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21132025-07-15T21:16:33.688ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0bbdeef7-f265-490b-b4b1-729384ad0bc5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21142025-07-15T21:16:33.689ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2052 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0bbdeef7-f265-490b-b4b1-729384ad0bc5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21152025-07-15T21:16:34.703ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7954d3ec-c3bf-47a7-9c95-c2b501c70df4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21162025-07-15T21:16:34.703ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7954d3ec-c3bf-47a7-9c95-c2b501c70df4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21172025-07-15T21:16:34.704ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1902 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7954d3ec-c3bf-47a7-9c95-c2b501c70df4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21182025-07-15T21:16:35.719ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9c9dd14c-67d3-48fc-a985-351b64928def roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21192025-07-15T21:16:35.719ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9c9dd14c-67d3-48fc-a985-351b64928def resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21202025-07-15T21:16:35.721ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1941 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9c9dd14c-67d3-48fc-a985-351b64928def response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21212025-07-15T21:16:36.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d622b736-739c-48c8-a228-51d1eff15241 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21222025-07-15T21:16:36.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d622b736-739c-48c8-a228-51d1eff15241 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21232025-07-15T21:16:36.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d622b736-739c-48c8-a228-51d1eff15241 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21242025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
21252025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
21262025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 125 reason = Timeout
21272025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21282025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21292025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
21302025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21312025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
21322025-07-15T21:16:36.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21332025-07-15T21:16:36.888ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21342025-07-15T21:16:36.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21352025-07-15T21:16:36.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21362025-07-15T21:16:36.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21372025-07-15T21:16:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
21382025-07-15T21:16:36.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
21392025-07-15T21:16:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21402025-07-15T21:16:36.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21412025-07-15T21:16:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
21422025-07-15T21:16:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21432025-07-15T21:16:36.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
21442025-07-15T21:16:36.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.298228ms iteration = 125 status = {"chicken_switches_updated":false}
21452025-07-15T21:16:36.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21462025-07-15T21:16:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21472025-07-15T21:16:36.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21482025-07-15T21:16:36.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
21492025-07-15T21:16:36.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21502025-07-15T21:16:36.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21512025-07-15T21:16:36.910ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
21522025-07-15T21:16:36.910ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.733673ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
21532025-07-15T21:16:36.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.550434ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21542025-07-15T21:16:37.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 96d0ea1c-87ba-46fb-9f67-d2309fb8a419 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21552025-07-15T21:16:37.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 96d0ea1c-87ba-46fb-9f67-d2309fb8a419 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21562025-07-15T21:16:37.755ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2584 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 96d0ea1c-87ba-46fb-9f67-d2309fb8a419 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21572025-07-15T21:16:38.768ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fa45a0b5-4b06-4920-8008-6a8279b10897 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21582025-07-15T21:16:38.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fa45a0b5-4b06-4920-8008-6a8279b10897 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21592025-07-15T21:16:38.770ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fa45a0b5-4b06-4920-8008-6a8279b10897 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21602025-07-15T21:16:39.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2b69cdd9-55f3-40e4-a579-1e143544a57c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21612025-07-15T21:16:39.785ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2b69cdd9-55f3-40e4-a579-1e143544a57c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21622025-07-15T21:16:39.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2b69cdd9-55f3-40e4-a579-1e143544a57c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21632025-07-15T21:16:39.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 469 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 8c4e6cf6-f0bc-4a2b-96ef-724b00490699 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
21642025-07-15T21:16:40.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 85742af6-90bd-4bf4-9ba7-858beadfb19d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21652025-07-15T21:16:40.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 85742af6-90bd-4bf4-9ba7-858beadfb19d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21662025-07-15T21:16:40.802ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 85742af6-90bd-4bf4-9ba7-858beadfb19d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21672025-07-15T21:16:41.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3507f49d-e836-4ea4-bee5-29d11f84a1d3 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21682025-07-15T21:16:41.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3507f49d-e836-4ea4-bee5-29d11f84a1d3 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21692025-07-15T21:16:41.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2316 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3507f49d-e836-4ea4-bee5-29d11f84a1d3 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21702025-07-15T21:16:41.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 126 reason = Timeout
21712025-07-15T21:16:41.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
21722025-07-15T21:16:41.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
21732025-07-15T21:16:41.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21742025-07-15T21:16:41.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
21752025-07-15T21:16:41.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
21762025-07-15T21:16:41.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
21772025-07-15T21:16:41.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
21782025-07-15T21:16:41.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.287283ms iteration = 126 status = {"chicken_switches_updated":false}
21792025-07-15T21:16:42.832ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5ff41149-676a-41ab-aaad-7c269cc1e6f1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21802025-07-15T21:16:42.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5ff41149-676a-41ab-aaad-7c269cc1e6f1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21812025-07-15T21:16:42.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2265 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5ff41149-676a-41ab-aaad-7c269cc1e6f1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21822025-07-15T21:16:43.848ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4726a425-55f6-4f63-8bc4-4b5665266448 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21832025-07-15T21:16:43.848ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4726a425-55f6-4f63-8bc4-4b5665266448 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21842025-07-15T21:16:43.850ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1933 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4726a425-55f6-4f63-8bc4-4b5665266448 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21852025-07-15T21:16:44.864ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 40cf6d8d-d8fd-450f-95ae-8101e9adfe7f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21862025-07-15T21:16:44.864ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 40cf6d8d-d8fd-450f-95ae-8101e9adfe7f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21872025-07-15T21:16:44.866ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2084 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 40cf6d8d-d8fd-450f-95ae-8101e9adfe7f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21882025-07-15T21:16:45.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ed2a924d-0f15-46e1-8ba8-ea91506bc94b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21892025-07-15T21:16:45.881ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ed2a924d-0f15-46e1-8ba8-ea91506bc94b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21902025-07-15T21:16:45.883ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2029 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ed2a924d-0f15-46e1-8ba8-ea91506bc94b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
21912025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
21922025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
21932025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21942025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
21952025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21962025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21972025-07-15T21:16:46.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21982025-07-15T21:16:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 307120747 } })
21992025-07-15T21:16:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 307301292 } })
22002025-07-15T21:16:46.817ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
22012025-07-15T21:16:46.817ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
22022025-07-15T21:16:46.818ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.612663ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
22032025-07-15T21:16:46.855ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
22042025-07-15T21:16:46.855ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22052025-07-15T21:16:46.856ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "df9bcb06-0a7a-4739-9b49-f16e443b6c88", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
22062025-07-15T21:16:46.856ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22072025-07-15T21:16:46.856ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
22082025-07-15T21:16:46.856ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22092025-07-15T21:16:46.856ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "485c13b5-d363-4189-a9df-131aa2317b66", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
22102025-07-15T21:16:46.856ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22112025-07-15T21:16:46.857ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22122025-07-15T21:16:46.859ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22132025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
22142025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
22152025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
22162025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22172025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
22182025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
22192025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
22202025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22212025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22222025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
22232025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 127 reason = Timeout
22242025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
22252025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22262025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
22272025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
22282025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22292025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22302025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
22312025-07-15T21:16:46.886ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
22322025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22332025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
22342025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22352025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
22362025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
22372025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
22382025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
22392025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22402025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22412025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
22422025-07-15T21:16:46.887ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
22432025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
22442025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22452025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22462025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22472025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
22482025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
22492025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
22502025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22512025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
22522025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22532025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
22542025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22552025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
22562025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22572025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22582025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22592025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
22602025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22612025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22622025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22632025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22642025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22652025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22662025-07-15T21:16:46.887ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 377759542 } })
22672025-07-15T21:16:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22682025-07-15T21:16:46.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22692025-07-15T21:16:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22702025-07-15T21:16:46.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22712025-07-15T21:16:46.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22722025-07-15T21:16:46.892ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22732025-07-15T21:16:46.895ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.12602ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
22742025-07-15T21:16:46.895ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22752025-07-15T21:16:46.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
22762025-07-15T21:16:46.896ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22772025-07-15T21:16:46.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22782025-07-15T21:16:46.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22792025-07-15T21:16:46.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 22db9ddf-f323-46cc-8de0-1a1a3a883bcb roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
22802025-07-15T21:16:46.897ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22812025-07-15T21:16:46.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22822025-07-15T21:16:46.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22832025-07-15T21:16:46.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 22db9ddf-f323-46cc-8de0-1a1a3a883bcb resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
22842025-07-15T21:16:46.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22852025-07-15T21:16:46.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22862025-07-15T21:16:46.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22872025-07-15T21:16:46.898ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22882025-07-15T21:16:46.899ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22892025-07-15T21:16:46.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22902025-07-15T21:16:46.900ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 13.56389ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22912025-07-15T21:16:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
22922025-07-15T21:16:46.900ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
22932025-07-15T21:16:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
22942025-07-15T21:16:46.900ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
22952025-07-15T21:16:46.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22962025-07-15T21:16:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22972025-07-15T21:16:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22982025-07-15T21:16:46.901ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22992025-07-15T21:16:46.901ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 4788 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 22db9ddf-f323-46cc-8de0-1a1a3a883bcb response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
23002025-07-15T21:16:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23012025-07-15T21:16:46.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
23022025-07-15T21:16:46.902ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23032025-07-15T21:16:46.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23042025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23052025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23062025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
23072025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 16.673361ms iteration = 127 status = {"chicken_switches_updated":false}
23082025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23092025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
23102025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23112025-07-15T21:16:46.903ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23122025-07-15T21:16:46.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23132025-07-15T21:16:46.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23142025-07-15T21:16:46.904ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23152025-07-15T21:16:46.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23162025-07-15T21:16:46.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23172025-07-15T21:16:46.906ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1614, tv_nsec: 396366303 } })
23182025-07-15T21:16:46.906ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
23192025-07-15T21:16:46.906ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23202025-07-15T21:16:46.906ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23212025-07-15T21:16:46.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23222025-07-15T21:16:46.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
23232025-07-15T21:16:46.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23242025-07-15T21:16:46.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23252025-07-15T21:16:46.926ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
23262025-07-15T21:16:46.926ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.028674ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
23272025-07-15T21:16:46.946ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23282025-07-15T21:16:46.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23292025-07-15T21:16:46.948ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23302025-07-15T21:16:46.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23312025-07-15T21:16:46.951ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23322025-07-15T21:16:46.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23332025-07-15T21:16:46.951ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 64.897117ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
23342025-07-15T21:16:46.952ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 66.296633ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
23352025-07-15T21:16:46.954ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23362025-07-15T21:16:46.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23372025-07-15T21:16:46.956ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23382025-07-15T21:16:46.956ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
23392025-07-15T21:16:46.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23402025-07-15T21:16:46.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23412025-07-15T21:16:46.957ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23422025-07-15T21:16:46.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23432025-07-15T21:16:46.959ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23442025-07-15T21:16:46.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23452025-07-15T21:16:46.964ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23462025-07-15T21:16:46.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23472025-07-15T21:16:46.971ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23482025-07-15T21:16:46.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23492025-07-15T21:16:46.974ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23502025-07-15T21:16:46.974ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
23512025-07-15T21:16:46.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23522025-07-15T21:16:46.974ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 87.425479ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
23532025-07-15T21:16:46.986ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23542025-07-15T21:16:46.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23552025-07-15T21:16:46.992ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 105.433885ms iteration = 25 status = {"physical_disks_added":0}
23562025-07-15T21:16:46.998ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23572025-07-15T21:16:46.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23582025-07-15T21:16:47.000ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 185.604336ms iteration = 22 status = null
23592025-07-15T21:16:47.005ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 118.02675ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23602025-07-15T21:16:47.009ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23612025-07-15T21:16:47.012ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 125.411611ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
23622025-07-15T21:16:47.012ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23632025-07-15T21:16:47.012ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23642025-07-15T21:16:47.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23652025-07-15T21:16:47.013ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 126.323944ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23662025-07-15T21:16:47.014ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23672025-07-15T21:16:47.014ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
23682025-07-15T21:16:47.014ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23692025-07-15T21:16:47.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23702025-07-15T21:16:47.017ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23712025-07-15T21:16:47.017ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
23722025-07-15T21:16:47.017ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 130.410899ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
23732025-07-15T21:16:47.020ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23742025-07-15T21:16:47.023ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
23752025-07-15T21:16:47.023ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23762025-07-15T21:16:47.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23772025-07-15T21:16:47.026ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 140.012208ms iteration = 22 status = 0
23782025-07-15T21:16:47.029ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
23792025-07-15T21:16:47.029ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23802025-07-15T21:16:47.029ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "22532709-da41-4e51-a410-565ec22829b2", "content-length": "26", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
23812025-07-15T21:16:47.029ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23822025-07-15T21:16:47.030ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 215.923849ms iteration = 22 status = {}
23832025-07-15T21:16:47.031ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
23842025-07-15T21:16:47.046ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23852025-07-15T21:16:47.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23862025-07-15T21:16:47.048ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23872025-07-15T21:16:47.050ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23882025-07-15T21:16:47.050ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23892025-07-15T21:16:47.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23902025-07-15T21:16:47.051ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
23912025-07-15T21:16:47.051ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 164.200293ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
23922025-07-15T21:16:47.052ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23932025-07-15T21:16:47.054ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23942025-07-15T21:16:47.054ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23952025-07-15T21:16:47.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23962025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
23972025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "b146db68-12c8-46ef-bc69-b6ae0ff530c6", "content-length": "78", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
23982025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
23992025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
24002025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
24012025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
24022025-07-15T21:16:47.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
24032025-07-15T21:16:47.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
24042025-07-15T21:16:47.087ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
24052025-07-15T21:16:47.088ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
24062025-07-15T21:16:47.088ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
24072025-07-15T21:16:47.089ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
24082025-07-15T21:16:47.089ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
24092025-07-15T21:16:47.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
24102025-07-15T21:16:47.090ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
24112025-07-15T21:16:47.090ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
24122025-07-15T21:16:47.090ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
24132025-07-15T21:16:47.090ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
24142025-07-15T21:16:47.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
24152025-07-15T21:16:47.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
24162025-07-15T21:16:47.093ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
24172025-07-15T21:16:47.093ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
24182025-07-15T21:16:47.094ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "dda59746-e980-4cf0-bd24-2019d34aeb79", "content-length": "81", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
24192025-07-15T21:16:47.129ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
24202025-07-15T21:16:47.129ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
24212025-07-15T21:16:47.129ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d27d964-6aa6-4f41-8105-4e02f10efb20", "content-length": "81", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
24222025-07-15T21:16:47.132ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
24232025-07-15T21:16:47.132ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
24242025-07-15T21:16:47.133ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9ac03ac-b575-4024-bb1c-240796f50d03", "content-length": "81", "date": "Tue, 15 Jul 2025 21:16:46 GMT"} })
24252025-07-15T21:16:47.134ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 247.506776ms iteration = 22 status = {"errors":[],"sps":[]}
24262025-07-15T21:16:47.184ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
24272025-07-15T21:16:47.186ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "94d6d472-a041-4809-bb3d-c8e8643d3e2a", "content-length": "691", "date": "Tue, 15 Jul 2025 21:16:47 GMT"} })
24282025-07-15T21:16:47.186ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
24292025-07-15T21:16:47.186ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
24302025-07-15T21:16:47.186ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
24312025-07-15T21:16:47.188ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 301.808239ms iteration = 22 status = {}
24322025-07-15T21:16:47.232ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
24332025-07-15T21:16:47.232ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24342025-07-15T21:16:47.233ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d82f4c3-9348-4a03-8a99-bd5e04a16e01", "content-length": "372", "date": "Tue, 15 Jul 2025 21:16:47 GMT"} })
24352025-07-15T21:16:47.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24362025-07-15T21:16:47.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24372025-07-15T21:16:47.283ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
24382025-07-15T21:16:47.283ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
24392025-07-15T21:16:47.283ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4ad868b5-4f63-4c05-a8c9-ae73e979c4a6", "date": "Tue, 15 Jul 2025 21:16:47 GMT"} })
24402025-07-15T21:16:47.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24412025-07-15T21:16:47.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24422025-07-15T21:16:47.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24432025-07-15T21:16:47.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24442025-07-15T21:16:47.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24452025-07-15T21:16:47.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24462025-07-15T21:16:47.293ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
24472025-07-15T21:16:47.293ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24482025-07-15T21:16:47.293ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c3bfe284-0c04-4577-bdfc-95261f22deda", "date": "Tue, 15 Jul 2025 21:16:47 GMT"} })
24492025-07-15T21:16:47.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24502025-07-15T21:16:47.295ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 408.559337ms iteration = 22 status = {}
24512025-07-15T21:16:47.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24522025-07-15T21:16:47.297ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
24532025-07-15T21:16:47.297ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24542025-07-15T21:16:47.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24552025-07-15T21:16:47.329ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
24562025-07-15T21:16:47.329ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
24572025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
24582025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
24592025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24602025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24612025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24622025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24632025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24642025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24652025-07-15T21:16:47.330ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
24662025-07-15T21:16:47.330ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24672025-07-15T21:16:47.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24682025-07-15T21:16:47.334ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
24692025-07-15T21:16:47.334ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24702025-07-15T21:16:47.474ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "aad17eab-6990-42b9-b44e-c757673ccd81", "date": "Tue, 15 Jul 2025 21:16:47 GMT"} })
24712025-07-15T21:16:47.474ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
24722025-07-15T21:16:47.474ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24732025-07-15T21:16:47.475ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e718161-8c82-4886-93d1-1b3e57f82a6b", "content-length": "682", "date": "Tue, 15 Jul 2025 21:16:47 GMT"} })
24742025-07-15T21:16:47.475ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24752025-07-15T21:16:47.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24762025-07-15T21:16:47.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24772025-07-15T21:16:47.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24782025-07-15T21:16:47.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24792025-07-15T21:16:47.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24802025-07-15T21:16:47.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24812025-07-15T21:16:47.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24822025-07-15T21:16:47.491ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 604.673855ms iteration = 22 status = {}
24832025-07-15T21:16:47.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fab73342-bf2c-422a-92cc-bc272dc7c962 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24842025-07-15T21:16:47.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fab73342-bf2c-422a-92cc-bc272dc7c962 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24852025-07-15T21:16:47.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2062 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fab73342-bf2c-422a-92cc-bc272dc7c962 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24862025-07-15T21:16:48.936ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 525c3f64-6937-47d7-a062-fe5fb46f9df0 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24872025-07-15T21:16:48.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 525c3f64-6937-47d7-a062-fe5fb46f9df0 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24882025-07-15T21:16:48.938ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2140 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 525c3f64-6937-47d7-a062-fe5fb46f9df0 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24892025-07-15T21:16:49.953ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 73657a4a-237b-409b-a351-82f60081a471 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24902025-07-15T21:16:49.953ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 73657a4a-237b-409b-a351-82f60081a471 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24912025-07-15T21:16:49.955ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2186 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 73657a4a-237b-409b-a351-82f60081a471 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24922025-07-15T21:16:49.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 529 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 097f0a72-7e4b-4923-af27-6170cd5d077f response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
24932025-07-15T21:16:50.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9296f14d-3040-4255-a848-60dde5921cff roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24942025-07-15T21:16:50.969ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9296f14d-3040-4255-a848-60dde5921cff resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24952025-07-15T21:16:50.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2302 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9296f14d-3040-4255-a848-60dde5921cff response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
24962025-07-15T21:16:51.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 128 reason = Timeout
24972025-07-15T21:16:51.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24982025-07-15T21:16:51.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24992025-07-15T21:16:51.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25002025-07-15T21:16:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
25012025-07-15T21:16:51.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25022025-07-15T21:16:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25032025-07-15T21:16:51.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
25042025-07-15T21:16:51.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.693433ms iteration = 128 status = {"chicken_switches_updated":false}
25052025-07-15T21:16:51.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cf0e6b41-99fe-49d5-adaf-864e4b91ef5f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25062025-07-15T21:16:51.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cf0e6b41-99fe-49d5-adaf-864e4b91ef5f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25072025-07-15T21:16:51.988ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2122 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cf0e6b41-99fe-49d5-adaf-864e4b91ef5f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25082025-07-15T21:16:53.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7c2d6cf5-130f-466e-aa18-672348cee3ae roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25092025-07-15T21:16:53.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7c2d6cf5-130f-466e-aa18-672348cee3ae resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25102025-07-15T21:16:53.003ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2217 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7c2d6cf5-130f-466e-aa18-672348cee3ae response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25112025-07-15T21:16:54.017ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4bdeea9b-3c77-4eb2-a3d9-237f6463b0b5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25122025-07-15T21:16:54.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4bdeea9b-3c77-4eb2-a3d9-237f6463b0b5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25132025-07-15T21:16:54.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2058 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4bdeea9b-3c77-4eb2-a3d9-237f6463b0b5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25142025-07-15T21:16:55.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8aab5d89-2929-4bcf-8e0a-8c6b207f01af roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25152025-07-15T21:16:55.035ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8aab5d89-2929-4bcf-8e0a-8c6b207f01af resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25162025-07-15T21:16:55.037ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2217 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8aab5d89-2929-4bcf-8e0a-8c6b207f01af response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25172025-07-15T21:16:56.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e789b609-71b4-4ea4-a8d2-acc0c8686334 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25182025-07-15T21:16:56.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e789b609-71b4-4ea4-a8d2-acc0c8686334 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25192025-07-15T21:16:56.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1852 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e789b609-71b4-4ea4-a8d2-acc0c8686334 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25202025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
25212025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25222025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
25232025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 129 reason = Timeout
25242025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25252025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25262025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25272025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25282025-07-15T21:16:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25292025-07-15T21:16:56.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25302025-07-15T21:16:56.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25312025-07-15T21:16:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25322025-07-15T21:16:56.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25332025-07-15T21:16:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
25342025-07-15T21:16:56.892ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25352025-07-15T21:16:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25362025-07-15T21:16:56.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25372025-07-15T21:16:56.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25382025-07-15T21:16:56.894ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
25392025-07-15T21:16:56.894ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.404637ms iteration = 129 status = {"chicken_switches_updated":false}
25402025-07-15T21:16:56.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25412025-07-15T21:16:56.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25422025-07-15T21:16:56.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
25432025-07-15T21:16:56.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25442025-07-15T21:16:56.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25452025-07-15T21:16:56.912ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
25462025-07-15T21:16:56.912ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.493825ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
25472025-07-15T21:16:56.921ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25482025-07-15T21:16:56.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25492025-07-15T21:16:56.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.560144ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25502025-07-15T21:16:57.068ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a64b433b-88ba-48e7-b1d9-9f57b049e404 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25512025-07-15T21:16:57.068ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a64b433b-88ba-48e7-b1d9-9f57b049e404 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25522025-07-15T21:16:57.070ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a64b433b-88ba-48e7-b1d9-9f57b049e404 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25532025-07-15T21:16:58.084ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e2db3476-5a68-42a3-9165-d6c759e9ddf4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25542025-07-15T21:16:58.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e2db3476-5a68-42a3-9165-d6c759e9ddf4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25552025-07-15T21:16:58.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1981 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e2db3476-5a68-42a3-9165-d6c759e9ddf4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25562025-07-15T21:16:59.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dec7cc5c-3732-4a3d-896a-bdb71f16e11c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25572025-07-15T21:16:59.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dec7cc5c-3732-4a3d-896a-bdb71f16e11c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25582025-07-15T21:16:59.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2069 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dec7cc5c-3732-4a3d-896a-bdb71f16e11c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25592025-07-15T21:16:59.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 467 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 907e779a-7874-48a5-87c8-7d82c72c5661 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
25602025-07-15T21:17:00.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad141996-2589-487d-b23e-c8e8c3ac395f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25612025-07-15T21:17:00.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad141996-2589-487d-b23e-c8e8c3ac395f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25622025-07-15T21:17:00.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2494 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad141996-2589-487d-b23e-c8e8c3ac395f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25632025-07-15T21:17:01.133ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0bfc495d-a854-4330-9cb8-caebce50732b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25642025-07-15T21:17:01.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0bfc495d-a854-4330-9cb8-caebce50732b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25652025-07-15T21:17:01.135ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1939 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0bfc495d-a854-4330-9cb8-caebce50732b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25662025-07-15T21:17:01.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 130 reason = Timeout
25672025-07-15T21:17:01.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25682025-07-15T21:17:01.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25692025-07-15T21:17:01.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25702025-07-15T21:17:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
25712025-07-15T21:17:01.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25722025-07-15T21:17:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25732025-07-15T21:17:01.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
25742025-07-15T21:17:01.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.486788ms iteration = 130 status = {"chicken_switches_updated":false}
25752025-07-15T21:17:02.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b112b4cc-1819-44c8-b177-ebc60796e3cd roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25762025-07-15T21:17:02.150ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b112b4cc-1819-44c8-b177-ebc60796e3cd resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25772025-07-15T21:17:02.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2546 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b112b4cc-1819-44c8-b177-ebc60796e3cd response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25782025-07-15T21:17:03.166ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d99d1005-06e5-4cc1-8532-c6e5323b7387 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25792025-07-15T21:17:03.167ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d99d1005-06e5-4cc1-8532-c6e5323b7387 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25802025-07-15T21:17:03.168ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d99d1005-06e5-4cc1-8532-c6e5323b7387 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25812025-07-15T21:17:04.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca11e210-4fac-41b0-97d8-6bafdb360367 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25822025-07-15T21:17:04.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca11e210-4fac-41b0-97d8-6bafdb360367 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25832025-07-15T21:17:04.185ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2320 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca11e210-4fac-41b0-97d8-6bafdb360367 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25842025-07-15T21:17:05.200ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ec006316-fd72-40f2-8dd7-7c4fbb029068 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25852025-07-15T21:17:05.200ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ec006316-fd72-40f2-8dd7-7c4fbb029068 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25862025-07-15T21:17:05.202ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2113 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ec006316-fd72-40f2-8dd7-7c4fbb029068 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25872025-07-15T21:17:06.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0ca31155-8c0f-4d7c-91ec-10c3323f7c53 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25882025-07-15T21:17:06.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0ca31155-8c0f-4d7c-91ec-10c3323f7c53 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25892025-07-15T21:17:06.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1767 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0ca31155-8c0f-4d7c-91ec-10c3323f7c53 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
25902025-07-15T21:17:06.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
25912025-07-15T21:17:06.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
25922025-07-15T21:17:06.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 131 reason = Timeout
25932025-07-15T21:17:06.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25942025-07-15T21:17:06.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25952025-07-15T21:17:06.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25962025-07-15T21:17:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25972025-07-15T21:17:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25982025-07-15T21:17:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25992025-07-15T21:17:06.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26002025-07-15T21:17:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26012025-07-15T21:17:06.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26022025-07-15T21:17:06.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26032025-07-15T21:17:06.889ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.687865ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26042025-07-15T21:17:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
26052025-07-15T21:17:06.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
26062025-07-15T21:17:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26072025-07-15T21:17:06.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26082025-07-15T21:17:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
26092025-07-15T21:17:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26102025-07-15T21:17:06.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26112025-07-15T21:17:06.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
26122025-07-15T21:17:06.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.899754ms iteration = 131 status = {"chicken_switches_updated":false}
26132025-07-15T21:17:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26142025-07-15T21:17:06.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26152025-07-15T21:17:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
26162025-07-15T21:17:06.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26172025-07-15T21:17:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26182025-07-15T21:17:06.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
26192025-07-15T21:17:06.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.340283ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
26202025-07-15T21:17:07.232ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 37f88e1b-09ef-437e-9b4e-714d1f3d2ee5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26212025-07-15T21:17:07.232ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 37f88e1b-09ef-437e-9b4e-714d1f3d2ee5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26222025-07-15T21:17:07.234ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2400 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 37f88e1b-09ef-437e-9b4e-714d1f3d2ee5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26232025-07-15T21:17:08.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = abb9aa83-e5d9-4e64-ba7a-5e2dda152b71 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26242025-07-15T21:17:08.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = abb9aa83-e5d9-4e64-ba7a-5e2dda152b71 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26252025-07-15T21:17:08.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2700 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = abb9aa83-e5d9-4e64-ba7a-5e2dda152b71 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26262025-07-15T21:17:09.266ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1ac2754e-cf4d-493e-8af1-ded9f4c76787 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26272025-07-15T21:17:09.266ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1ac2754e-cf4d-493e-8af1-ded9f4c76787 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26282025-07-15T21:17:09.268ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2687 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1ac2754e-cf4d-493e-8af1-ded9f4c76787 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26292025-07-15T21:17:09.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 470 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 7ed32b5c-94b4-468c-a9b9-71b5efd8301b response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
26302025-07-15T21:17:10.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f31604ea-e20b-4ea2-a58e-de61628a4f84 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26312025-07-15T21:17:10.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f31604ea-e20b-4ea2-a58e-de61628a4f84 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26322025-07-15T21:17:10.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2071 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f31604ea-e20b-4ea2-a58e-de61628a4f84 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26332025-07-15T21:17:11.299ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 67620eb7-7e62-4d5c-b798-95196837e2d2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26342025-07-15T21:17:11.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 67620eb7-7e62-4d5c-b798-95196837e2d2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26352025-07-15T21:17:11.301ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1985 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 67620eb7-7e62-4d5c-b798-95196837e2d2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26362025-07-15T21:17:11.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 132 reason = Timeout
26372025-07-15T21:17:11.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
26382025-07-15T21:17:11.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
26392025-07-15T21:17:11.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26402025-07-15T21:17:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
26412025-07-15T21:17:11.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
26422025-07-15T21:17:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
26432025-07-15T21:17:11.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
26442025-07-15T21:17:11.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.313703ms iteration = 132 status = {"chicken_switches_updated":false}
26452025-07-15T21:17:12.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9b4baa0f-d315-4896-a8ee-0c23b633a05f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26462025-07-15T21:17:12.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9b4baa0f-d315-4896-a8ee-0c23b633a05f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26472025-07-15T21:17:12.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2503 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9b4baa0f-d315-4896-a8ee-0c23b633a05f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26482025-07-15T21:17:13.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1a15025a-8f22-4200-b2ba-d8f11f6578e6 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26492025-07-15T21:17:13.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1a15025a-8f22-4200-b2ba-d8f11f6578e6 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26502025-07-15T21:17:13.334ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2412 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1a15025a-8f22-4200-b2ba-d8f11f6578e6 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26512025-07-15T21:17:14.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 04ab2d2c-87dd-4792-ba9a-b1a19494f032 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26522025-07-15T21:17:14.350ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 04ab2d2c-87dd-4792-ba9a-b1a19494f032 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26532025-07-15T21:17:14.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 04ab2d2c-87dd-4792-ba9a-b1a19494f032 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26542025-07-15T21:17:15.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8705938c-c61f-4307-bb0a-028a67b8ae25 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26552025-07-15T21:17:15.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8705938c-c61f-4307-bb0a-028a67b8ae25 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26562025-07-15T21:17:15.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2088 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8705938c-c61f-4307-bb0a-028a67b8ae25 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26572025-07-15T21:17:16.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad2fcdd0-7894-43ac-9ae0-82f8b29ff9f6 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26582025-07-15T21:17:16.384ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad2fcdd0-7894-43ac-9ae0-82f8b29ff9f6 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26592025-07-15T21:17:16.385ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2013 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ad2fcdd0-7894-43ac-9ae0-82f8b29ff9f6 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
26602025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
26612025-07-15T21:17:16.815ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225
26622025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
26632025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 27.061\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225"}
26642025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
26652025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26662025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
26672025-07-15T21:17:16.815ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-07-15 21:07:16.815051580 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
26682025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
26692025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
26702025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
26712025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
26722025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26732025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
26742025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
26752025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26762025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
26772025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26782025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
26792025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
26802025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26812025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
26822025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
26832025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26842025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26852025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26862025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26872025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
26882025-07-15T21:17:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
26892025-07-15T21:17:16.815ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4fcf02c2-2876-4bd3-b6eb-3250110a4c40.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 306035739 } })
26902025-07-15T21:17:16.815ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ada38bf-5331-48b0-9340-612e7eca7fae.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1fb7ee6e-8db0-4006-be77-f524ca0c97d3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 306052200 } })
26912025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26922025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
26932025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.229761ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
26942025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
26952025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.132869ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
26962025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26972025-07-15T21:17:16.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26982025-07-15T21:17:16.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26992025-07-15T21:17:16.818ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 308452582 } })
27002025-07-15T21:17:16.818ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
27012025-07-15T21:17:16.818ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27022025-07-15T21:17:16.819ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.912041ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
27032025-07-15T21:17:16.819ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27042025-07-15T21:17:16.819ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-07-15 21:07:16.815051580 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
27052025-07-15T21:17:16.819ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.895266ms iteration = 12 status = {"expiration":"2025-07-15T21:07:16.815051580Z","pruned":[]}
27062025-07-15T21:17:16.819ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 310225418 } })
27072025-07-15T21:17:16.820ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
27082025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Timeout
27092025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
27102025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 6.06\u{b5}s iteration = 16 status = "Disabled"
27112025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
27122025-07-15T21:17:16.820ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
27132025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
27142025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
27152025-07-15T21:17:16.820ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27162025-07-15T21:17:16.821ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
27172025-07-15T21:17:16.821ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27182025-07-15T21:17:16.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27192025-07-15T21:17:16.822ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27202025-07-15T21:17:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27212025-07-15T21:17:16.824ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27222025-07-15T21:17:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27232025-07-15T21:17:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27242025-07-15T21:17:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
27252025-07-15T21:17:16.824ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
27262025-07-15T21:17:16.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
27272025-07-15T21:17:16.825ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27282025-07-15T21:17:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
27292025-07-15T21:17:16.829ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27302025-07-15T21:17:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27312025-07-15T21:17:16.830ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
27322025-07-15T21:17:16.830ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
27332025-07-15T21:17:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
27342025-07-15T21:17:16.830ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
27352025-07-15T21:17:16.830ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
27362025-07-15T21:17:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
27372025-07-15T21:17:16.832ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
27382025-07-15T21:17:16.832ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
27392025-07-15T21:17:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
27402025-07-15T21:17:16.834ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
27412025-07-15T21:17:16.834ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
27422025-07-15T21:17:16.834ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
27432025-07-15T21:17:16.834ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
27442025-07-15T21:17:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
27452025-07-15T21:17:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
27462025-07-15T21:17:16.837ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.577338ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
27472025-07-15T21:17:16.841ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
27482025-07-15T21:17:16.841ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
27492025-07-15T21:17:16.841ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 11.711842ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
27502025-07-15T21:17:16.842ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
27512025-07-15T21:17:16.842ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 8.50089ms iteration = 12 status = {"by_rx":{},"error":null}
27522025-07-15T21:17:16.845ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
27532025-07-15T21:17:16.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
27542025-07-15T21:17:16.848ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 14.69678ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"162.0.0","globs_reprocessed":{},"no_receivers":[]}
27552025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
27562025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
27572025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
27582025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27592025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
27602025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
27612025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27622025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 133 reason = Timeout
27632025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27642025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
27652025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
27662025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
27672025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27682025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27692025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27702025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
27712025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27722025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
27732025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
27742025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27752025-07-15T21:17:16.885ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27762025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27772025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
27782025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
27792025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27802025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27812025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27822025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27832025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27842025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27852025-07-15T21:17:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27862025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
27872025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
27882025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
27892025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27902025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
27912025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
27922025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27932025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
27942025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27952025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
27962025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
27972025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
27982025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27992025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
28002025-07-15T21:17:16.887ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28012025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28022025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
28032025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28042025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28052025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28062025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
28072025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
28082025-07-15T21:17:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28092025-07-15T21:17:16.887ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 378104481 } })
28102025-07-15T21:17:16.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28112025-07-15T21:17:16.889ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28122025-07-15T21:17:16.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28132025-07-15T21:17:16.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28142025-07-15T21:17:16.892ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28152025-07-15T21:17:16.892ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28162025-07-15T21:17:16.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28172025-07-15T21:17:16.893ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28182025-07-15T21:17:16.893ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28192025-07-15T21:17:16.893ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
28202025-07-15T21:17:16.893ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28212025-07-15T21:17:16.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28222025-07-15T21:17:16.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28232025-07-15T21:17:16.894ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28242025-07-15T21:17:16.896ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28252025-07-15T21:17:16.896ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
28262025-07-15T21:17:16.896ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
28272025-07-15T21:17:16.896ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
28282025-07-15T21:17:16.896ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28292025-07-15T21:17:16.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28302025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28312025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28322025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28332025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28342025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28352025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28362025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
28372025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28382025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28392025-07-15T21:17:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28402025-07-15T21:17:16.898ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.170299ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
28412025-07-15T21:17:16.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28422025-07-15T21:17:16.898ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28432025-07-15T21:17:16.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28442025-07-15T21:17:16.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28452025-07-15T21:17:16.900ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 15.263635ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28462025-07-15T21:17:16.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28472025-07-15T21:17:16.901ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28482025-07-15T21:17:16.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28492025-07-15T21:17:16.902ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28502025-07-15T21:17:16.902ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 16.859756ms iteration = 133 status = {"chicken_switches_updated":false}
28512025-07-15T21:17:16.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
28522025-07-15T21:17:16.902ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28532025-07-15T21:17:16.902ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28542025-07-15T21:17:16.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28552025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
28562025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
28572025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
28582025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28592025-07-15T21:17:16.903ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28602025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
28612025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28622025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28632025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28642025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28652025-07-15T21:17:16.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28662025-07-15T21:17:16.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28672025-07-15T21:17:16.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28682025-07-15T21:17:16.905ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1644, tv_nsec: 395329577 } })
28692025-07-15T21:17:16.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28702025-07-15T21:17:16.905ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
28712025-07-15T21:17:16.906ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28722025-07-15T21:17:16.906ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
28732025-07-15T21:17:16.906ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
28742025-07-15T21:17:16.906ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 86.209488ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
28752025-07-15T21:17:16.906ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28762025-07-15T21:17:16.906ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28772025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28782025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28792025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28802025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28812025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28822025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28832025-07-15T21:17:16.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28842025-07-15T21:17:16.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28852025-07-15T21:17:16.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28862025-07-15T21:17:16.909ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28872025-07-15T21:17:16.911ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28882025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28892025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28902025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
28912025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
28922025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28932025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28942025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28952025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28962025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28972025-07-15T21:17:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28982025-07-15T21:17:16.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28992025-07-15T21:17:16.913ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29002025-07-15T21:17:16.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29012025-07-15T21:17:16.913ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
29022025-07-15T21:17:16.913ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.542426ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
29032025-07-15T21:17:16.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
29042025-07-15T21:17:16.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29052025-07-15T21:17:16.914ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29062025-07-15T21:17:16.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29072025-07-15T21:17:16.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29082025-07-15T21:17:16.915ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29092025-07-15T21:17:16.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29102025-07-15T21:17:16.917ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
29112025-07-15T21:17:16.917ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 0 version = 2
29122025-07-15T21:17:16.917ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29132025-07-15T21:17:16.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29142025-07-15T21:17:16.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29152025-07-15T21:17:16.919ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29162025-07-15T21:17:16.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29172025-07-15T21:17:16.919ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29182025-07-15T21:17:16.923ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
29192025-07-15T21:17:16.923ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
29202025-07-15T21:17:16.923ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-07-15 21:06:16.954295 UTC dns_group = internal generation = 1
29212025-07-15T21:17:16.923ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 108.476093ms iteration = 12 status = {"generation":1}
29222025-07-15T21:17:16.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29232025-07-15T21:17:16.925ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29242025-07-15T21:17:16.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29252025-07-15T21:17:16.933ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 4 version = 2
29262025-07-15T21:17:16.933ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
29272025-07-15T21:17:16.933ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-07-15 21:06:19.267830 UTC dns_group = external generation = 2
29282025-07-15T21:17:16.933ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 114.195551ms iteration = 12 status = {"generation":2}
29292025-07-15T21:17:16.952ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
29302025-07-15T21:17:16.952ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.054232ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
29312025-07-15T21:17:16.974ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29322025-07-15T21:17:16.975ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 87.777268ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
29332025-07-15T21:17:16.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29342025-07-15T21:17:16.975ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 89.601375ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
29352025-07-15T21:17:16.976ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29362025-07-15T21:17:16.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29372025-07-15T21:17:16.977ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29382025-07-15T21:17:16.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29392025-07-15T21:17:16.979ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29402025-07-15T21:17:16.980ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29412025-07-15T21:17:16.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29422025-07-15T21:17:16.983ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29432025-07-15T21:17:16.983ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29442025-07-15T21:17:16.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29452025-07-15T21:17:16.983ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29462025-07-15T21:17:16.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29472025-07-15T21:17:16.987ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29482025-07-15T21:17:16.987ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
29492025-07-15T21:17:16.987ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 101.958575ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
29502025-07-15T21:17:16.991ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29512025-07-15T21:17:16.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29522025-07-15T21:17:16.994ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29532025-07-15T21:17:16.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29542025-07-15T21:17:16.997ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29552025-07-15T21:17:17.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
29562025-07-15T21:17:17.002ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29572025-07-15T21:17:17.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29582025-07-15T21:17:17.008ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 121.288294ms iteration = 23 status = 0
29592025-07-15T21:17:17.010ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-07-15T21:06:12.596852Z, time_modified: 2025-07-15T21:06:12.596852Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }) }
29602025-07-15T21:17:17.010ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 195.07272ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]}},"default_endpoint":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
29612025-07-15T21:17:17.037ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
29622025-07-15T21:17:17.037ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 151.72776ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
29632025-07-15T21:17:17.039ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 153.474205ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
29642025-07-15T21:17:17.039ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
29652025-07-15T21:17:17.039ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29662025-07-15T21:17:17.040ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8460a612-f80a-40b8-8ae0-6a4b614e6d72", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
29672025-07-15T21:17:17.040ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29682025-07-15T21:17:17.042ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29692025-07-15T21:17:17.054ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 166.977765ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
29702025-07-15T21:17:17.054ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 167.085997ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
29712025-07-15T21:17:17.058ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
29722025-07-15T21:17:17.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
29732025-07-15T21:17:17.061ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
29742025-07-15T21:17:17.088ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
29752025-07-15T21:17:17.088ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "48519f20-9447-496f-8e29-2cd968a761a4", "content-length": "78", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
29762025-07-15T21:17:17.088ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
29772025-07-15T21:17:17.088ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
29782025-07-15T21:17:17.089ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
29792025-07-15T21:17:17.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
29802025-07-15T21:17:17.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
29812025-07-15T21:17:17.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
29822025-07-15T21:17:17.092ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
29832025-07-15T21:17:17.093ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
29842025-07-15T21:17:17.093ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
29852025-07-15T21:17:17.094ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
29862025-07-15T21:17:17.094ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
29872025-07-15T21:17:17.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
29882025-07-15T21:17:17.096ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
29892025-07-15T21:17:17.096ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
29902025-07-15T21:17:17.096ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
29912025-07-15T21:17:17.097ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
29922025-07-15T21:17:17.097ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
29932025-07-15T21:17:17.097ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29942025-07-15T21:17:17.097ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbef49ca-d457-447d-8374-58ce5d30ffb1", "content-length": "81", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
29952025-07-15T21:17:17.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
29962025-07-15T21:17:17.097ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
29972025-07-15T21:17:17.097ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
29982025-07-15T21:17:17.097ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "963629ed-9269-4f96-bc47-2e7699c723e9", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
29992025-07-15T21:17:17.097ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30002025-07-15T21:17:17.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
30012025-07-15T21:17:17.099ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
30022025-07-15T21:17:17.099ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
30032025-07-15T21:17:17.099ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
30042025-07-15T21:17:17.099ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "65ceaa25-434b-4ecd-8f8f-629cb4fa971a", "content-length": "81", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
30052025-07-15T21:17:17.106ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
30062025-07-15T21:17:17.106ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30072025-07-15T21:17:17.106ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "79c50b6a-0ade-4d47-bfe1-99b9548cc124", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
30082025-07-15T21:17:17.106ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30092025-07-15T21:17:17.108ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30102025-07-15T21:17:17.115ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30112025-07-15T21:17:17.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30122025-07-15T21:17:17.125ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30132025-07-15T21:17:17.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30142025-07-15T21:17:17.127ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30152025-07-15T21:17:17.130ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30162025-07-15T21:17:17.130ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30172025-07-15T21:17:17.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30182025-07-15T21:17:17.131ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30192025-07-15T21:17:17.134ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30202025-07-15T21:17:17.134ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30212025-07-15T21:17:17.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30222025-07-15T21:17:17.135ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
30232025-07-15T21:17:17.136ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30242025-07-15T21:17:17.136ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30252025-07-15T21:17:17.137ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8acba2ef-672f-47f2-8613-90124f803ca3", "content-length": "372", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
30262025-07-15T21:17:17.137ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30272025-07-15T21:17:17.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30282025-07-15T21:17:17.139ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
30292025-07-15T21:17:17.139ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
30302025-07-15T21:17:17.140ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c464f61-b08a-4968-b87d-f7ef83b7345b", "content-length": "81", "date": "Tue, 15 Jul 2025 21:17:16 GMT"} })
30312025-07-15T21:17:17.142ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 255.297967ms iteration = 23 status = {"errors":[],"sps":[]}
30322025-07-15T21:17:17.144ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 258.571412ms iteration = 26 status = {"physical_disks_added":0}
30332025-07-15T21:17:17.213ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
30342025-07-15T21:17:17.213ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30352025-07-15T21:17:17.214ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c6ce33c0-f028-402e-9f68-4b457dbfbc19", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
30362025-07-15T21:17:17.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30372025-07-15T21:17:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30382025-07-15T21:17:17.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30392025-07-15T21:17:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30402025-07-15T21:17:17.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30412025-07-15T21:17:17.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30422025-07-15T21:17:17.224ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
30432025-07-15T21:17:17.224ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30442025-07-15T21:17:17.225ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "42e9a185-10cf-4f14-b3cd-32b92c533f59", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
30452025-07-15T21:17:17.227ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 339.856862ms iteration = 23 status = {}
30462025-07-15T21:17:17.258ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
30472025-07-15T21:17:17.259ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 428.310417ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
30482025-07-15T21:17:17.275ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 459.875963ms iteration = 23 status = {}
30492025-07-15T21:17:17.312ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
30502025-07-15T21:17:17.313ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30512025-07-15T21:17:17.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30522025-07-15T21:17:17.324ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 509.509576ms iteration = 23 status = null
30532025-07-15T21:17:17.326ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30542025-07-15T21:17:17.327ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca345f5f-e266-4178-bd6e-e811c6744d55", "content-length": "691", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
30552025-07-15T21:17:17.327ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
30562025-07-15T21:17:17.327ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
30572025-07-15T21:17:17.327ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
30582025-07-15T21:17:17.329ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 443.443958ms iteration = 23 status = {}
30592025-07-15T21:17:17.335ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
30602025-07-15T21:17:17.336ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e7984f6b-64f0-4ae8-87d3-3cf3aff7c537", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30612025-07-15T21:17:17.336ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30622025-07-15T21:17:17.337ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "bc728e39-4e3f-4d10-9277-7b5715f33e1d", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30632025-07-15T21:17:17.337ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30642025-07-15T21:17:17.339ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30652025-07-15T21:17:17.339ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 524.494433ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
30662025-07-15T21:17:17.385ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
30672025-07-15T21:17:17.385ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2a44c167-3e39-4140-8839-bec7098ac00e", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30682025-07-15T21:17:17.385ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30692025-07-15T21:17:17.388ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2978a972-f4d9-4b40-8642-4a3a761d7cc5", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30702025-07-15T21:17:17.388ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30712025-07-15T21:17:17.390ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "4c3534bc-c96b-4a8c-8e0c-d3cacaff9845", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30722025-07-15T21:17:17.390ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30732025-07-15T21:17:17.393ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
30742025-07-15T21:17:17.393ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 578.383825ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
30752025-07-15T21:17:17.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca8949e3-2ffa-4bf2-bdec-323a33fc9a4b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
30762025-07-15T21:17:17.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca8949e3-2ffa-4bf2-bdec-323a33fc9a4b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
30772025-07-15T21:17:17.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3111 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ca8949e3-2ffa-4bf2-bdec-323a33fc9a4b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
30782025-07-15T21:17:17.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30792025-07-15T21:17:17.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30802025-07-15T21:17:17.431ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
30812025-07-15T21:17:17.431ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30822025-07-15T21:17:17.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30832025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
30842025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
30852025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
30862025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
30872025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30882025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30892025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30902025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30912025-07-15T21:17:17.453ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30922025-07-15T21:17:17.454ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30932025-07-15T21:17:17.454ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
30942025-07-15T21:17:17.454ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30952025-07-15T21:17:17.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30962025-07-15T21:17:17.457ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
30972025-07-15T21:17:17.457ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
30982025-07-15T21:17:17.605ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "09a5cef4-07bf-416e-94f5-f5cab8ced26b", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
30992025-07-15T21:17:17.605ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
31002025-07-15T21:17:17.605ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31012025-07-15T21:17:17.613ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f91d6ee9-6729-4efc-85fa-64a0d69fe5d4", "content-length": "682", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
31022025-07-15T21:17:17.613ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31032025-07-15T21:17:17.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31042025-07-15T21:17:17.618ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31052025-07-15T21:17:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31062025-07-15T21:17:17.622ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31072025-07-15T21:17:17.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31082025-07-15T21:17:17.624ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31092025-07-15T21:17:17.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31102025-07-15T21:17:17.632ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 746.876149ms iteration = 23 status = {}
31112025-07-15T21:17:17.781ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
31122025-07-15T21:17:17.781ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
31132025-07-15T21:17:17.817ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
31142025-07-15T21:17:17.817ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:63228
31152025-07-15T21:17:17.817ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:63228 req_id = 63a88b6a-13d3-455b-af2f-91dc061ea7d9 roles = RoleSet { roles: {} } uri = /metrics/producers
31162025-07-15T21:17:17.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:63228 req_id = 63a88b6a-13d3-455b-af2f-91dc061ea7d9 resource = Database result = Ok(()) uri = /metrics/producers
31172025-07-15T21:17:17.862ZINFOnexus: assigned collector to new producer collector_id = bf08f219-8402-4dbd-b274-8bb245b334b8 file = nexus/src/app/oximeter.rs:76 producer_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
31182025-07-15T21:17:17.862ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 45091 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:63228 req_id = 63a88b6a-13d3-455b-af2f-91dc061ea7d9 response_code = 201 uri = /metrics/producers
31192025-07-15T21:17:17.863ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "63a88b6a-13d3-455b-af2f-91dc061ea7d9", "content-length": "41", "date": "Tue, 15 Jul 2025 21:17:17 GMT"} })
31202025-07-15T21:17:17.865ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
31212025-07-15T21:17:18.422ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba60b689-b784-4be0-b9cb-c418d8f0506c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31222025-07-15T21:17:18.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba60b689-b784-4be0-b9cb-c418d8f0506c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31232025-07-15T21:17:18.424ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1917 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba60b689-b784-4be0-b9cb-c418d8f0506c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31242025-07-15T21:17:19.439ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2bb998d7-a1b9-4a9d-acf0-ab0c4c45b545 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31252025-07-15T21:17:19.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2bb998d7-a1b9-4a9d-acf0-ab0c4c45b545 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31262025-07-15T21:17:19.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2638 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2bb998d7-a1b9-4a9d-acf0-ab0c4c45b545 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31272025-07-15T21:17:19.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 480 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = da0edefa-4a30-4945-b10b-60b60fcb9079 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
31282025-07-15T21:17:20.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ee0e609a-f2d4-464d-b370-8e70c3b1e0ef roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31292025-07-15T21:17:20.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ee0e609a-f2d4-464d-b370-8e70c3b1e0ef resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31302025-07-15T21:17:20.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1979 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ee0e609a-f2d4-464d-b370-8e70c3b1e0ef response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31312025-07-15T21:17:21.474ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 38b1b649-84f6-478b-a1dc-071d33c216f3 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31322025-07-15T21:17:21.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 38b1b649-84f6-478b-a1dc-071d33c216f3 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31332025-07-15T21:17:21.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2610 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 38b1b649-84f6-478b-a1dc-071d33c216f3 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31342025-07-15T21:17:21.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 134 reason = Timeout
31352025-07-15T21:17:21.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31362025-07-15T21:17:21.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31372025-07-15T21:17:21.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31382025-07-15T21:17:21.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
31392025-07-15T21:17:21.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31402025-07-15T21:17:21.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31412025-07-15T21:17:21.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
31422025-07-15T21:17:21.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.349594ms iteration = 134 status = {"chicken_switches_updated":false}
31432025-07-15T21:17:22.490ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e7f739ad-435e-46c3-8e51-9fd718a68107 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31442025-07-15T21:17:22.490ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e7f739ad-435e-46c3-8e51-9fd718a68107 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31452025-07-15T21:17:22.493ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2715 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e7f739ad-435e-46c3-8e51-9fd718a68107 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31462025-07-15T21:17:23.507ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 765cab10-145f-4b7d-a6e8-69065dce3972 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31472025-07-15T21:17:23.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 765cab10-145f-4b7d-a6e8-69065dce3972 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31482025-07-15T21:17:23.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2130 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 765cab10-145f-4b7d-a6e8-69065dce3972 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31492025-07-15T21:17:24.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2038af8d-d81a-46fb-8521-fa5e88ee3b2e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31502025-07-15T21:17:24.524ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2038af8d-d81a-46fb-8521-fa5e88ee3b2e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31512025-07-15T21:17:24.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2681 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2038af8d-d81a-46fb-8521-fa5e88ee3b2e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31522025-07-15T21:17:25.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 73e0f336-9b88-4810-948b-73a8aac8bf2f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31532025-07-15T21:17:25.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 73e0f336-9b88-4810-948b-73a8aac8bf2f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31542025-07-15T21:17:25.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1962 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 73e0f336-9b88-4810-948b-73a8aac8bf2f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31552025-07-15T21:17:26.558ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5ea953f4-e0ad-4064-bcd4-d5121810cda2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31562025-07-15T21:17:26.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5ea953f4-e0ad-4064-bcd4-d5121810cda2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31572025-07-15T21:17:26.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1963 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5ea953f4-e0ad-4064-bcd4-d5121810cda2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31582025-07-15T21:17:26.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
31592025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31602025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
31612025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 135 reason = Timeout
31622025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31632025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31642025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31652025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31662025-07-15T21:17:26.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31672025-07-15T21:17:26.888ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31682025-07-15T21:17:26.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31692025-07-15T21:17:26.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31702025-07-15T21:17:26.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31712025-07-15T21:17:26.889ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.46931ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31722025-07-15T21:17:26.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
31732025-07-15T21:17:26.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31742025-07-15T21:17:26.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31752025-07-15T21:17:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31762025-07-15T21:17:26.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31772025-07-15T21:17:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31782025-07-15T21:17:26.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
31792025-07-15T21:17:26.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.009671ms iteration = 135 status = {"chicken_switches_updated":false}
31802025-07-15T21:17:26.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31812025-07-15T21:17:26.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31822025-07-15T21:17:26.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31832025-07-15T21:17:26.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
31842025-07-15T21:17:26.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31852025-07-15T21:17:26.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31862025-07-15T21:17:26.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
31872025-07-15T21:17:26.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.425229ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
31882025-07-15T21:17:27.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e87f1ecc-6fb4-49ce-a782-ce692eea6ae2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31892025-07-15T21:17:27.575ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e87f1ecc-6fb4-49ce-a782-ce692eea6ae2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31902025-07-15T21:17:27.577ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e87f1ecc-6fb4-49ce-a782-ce692eea6ae2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31912025-07-15T21:17:28.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 18e4613f-3e87-4675-bd06-a5db56433680 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31922025-07-15T21:17:28.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 18e4613f-3e87-4675-bd06-a5db56433680 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31932025-07-15T21:17:28.595ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3776 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 18e4613f-3e87-4675-bd06-a5db56433680 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31942025-07-15T21:17:29.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1d7ed287-6ce9-4b46-b8a8-7c29cdc4271d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31952025-07-15T21:17:29.618ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1d7ed287-6ce9-4b46-b8a8-7c29cdc4271d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31962025-07-15T21:17:29.621ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3712 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1d7ed287-6ce9-4b46-b8a8-7c29cdc4271d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31972025-07-15T21:17:29.977ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 700 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 097ec055-9fbc-4565-bcdd-f1a1a822c91e response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
31982025-07-15T21:17:30.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4505764c-c67d-45ec-b87a-836d9e455081 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
31992025-07-15T21:17:30.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4505764c-c67d-45ec-b87a-836d9e455081 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32002025-07-15T21:17:30.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3159 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4505764c-c67d-45ec-b87a-836d9e455081 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32012025-07-15T21:17:31.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2b0eb879-5f1b-4287-b28e-b804b6a04139 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32022025-07-15T21:17:31.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2b0eb879-5f1b-4287-b28e-b804b6a04139 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32032025-07-15T21:17:31.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2494 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2b0eb879-5f1b-4287-b28e-b804b6a04139 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32042025-07-15T21:17:31.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 136 reason = Timeout
32052025-07-15T21:17:31.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32062025-07-15T21:17:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32072025-07-15T21:17:31.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32082025-07-15T21:17:31.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
32092025-07-15T21:17:31.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32102025-07-15T21:17:31.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32112025-07-15T21:17:31.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
32122025-07-15T21:17:31.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.182119ms iteration = 136 status = {"chicken_switches_updated":false}
32132025-07-15T21:17:32.683ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b105298-ee22-4efb-b5f1-d92b14ad9df2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32142025-07-15T21:17:32.683ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b105298-ee22-4efb-b5f1-d92b14ad9df2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32152025-07-15T21:17:32.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2524 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b105298-ee22-4efb-b5f1-d92b14ad9df2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32162025-07-15T21:17:33.700ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d3c75f0c-6515-4464-b90a-fe0c15bd5c07 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32172025-07-15T21:17:33.701ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d3c75f0c-6515-4464-b90a-fe0c15bd5c07 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32182025-07-15T21:17:33.702ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2019 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d3c75f0c-6515-4464-b90a-fe0c15bd5c07 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32192025-07-15T21:17:34.718ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 46b8b1dd-3a07-4f72-9dd9-58f99d7bc57d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32202025-07-15T21:17:34.718ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 46b8b1dd-3a07-4f72-9dd9-58f99d7bc57d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32212025-07-15T21:17:34.720ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2482 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 46b8b1dd-3a07-4f72-9dd9-58f99d7bc57d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32222025-07-15T21:17:35.456ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2140 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a04e2387-39c6-40db-b3ea-45c81cd8ae8e uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2025-07-15T21:17:35.442116100Z }
32232025-07-15T21:17:35.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a04e2387-39c6-40db-b3ea-45c81cd8ae8e roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
32242025-07-15T21:17:35.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a04e2387-39c6-40db-b3ea-45c81cd8ae8e resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
32252025-07-15T21:17:35.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 34034 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a04e2387-39c6-40db-b3ea-45c81cd8ae8e response_code = 204 uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
32262025-07-15T21:17:35.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0dac5a7e-e194-4a6d-9730-e8aaed317542 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32272025-07-15T21:17:35.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0dac5a7e-e194-4a6d-9730-e8aaed317542 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32282025-07-15T21:17:35.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0dac5a7e-e194-4a6d-9730-e8aaed317542 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32292025-07-15T21:17:36.751ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 138c10f3-221b-4053-9f0a-742ff04da8df roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32302025-07-15T21:17:36.752ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 138c10f3-221b-4053-9f0a-742ff04da8df resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32312025-07-15T21:17:36.754ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2634 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 138c10f3-221b-4053-9f0a-742ff04da8df response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32322025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
32332025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32342025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
32352025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 137 reason = Timeout
32362025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32372025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32382025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32392025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32402025-07-15T21:17:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32412025-07-15T21:17:36.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32422025-07-15T21:17:36.888ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32432025-07-15T21:17:36.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32442025-07-15T21:17:36.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32452025-07-15T21:17:36.890ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.258216ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32462025-07-15T21:17:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
32472025-07-15T21:17:36.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32482025-07-15T21:17:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32492025-07-15T21:17:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32502025-07-15T21:17:36.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32512025-07-15T21:17:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32522025-07-15T21:17:36.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
32532025-07-15T21:17:36.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.388011ms iteration = 137 status = {"chicken_switches_updated":false}
32542025-07-15T21:17:36.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32552025-07-15T21:17:36.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32562025-07-15T21:17:36.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32572025-07-15T21:17:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
32582025-07-15T21:17:36.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32592025-07-15T21:17:36.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32602025-07-15T21:17:36.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
32612025-07-15T21:17:36.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.331551ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
32622025-07-15T21:17:37.768ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3cb26ad7-fe14-42ef-b550-50d980e72162 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32632025-07-15T21:17:37.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3cb26ad7-fe14-42ef-b550-50d980e72162 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32642025-07-15T21:17:37.770ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2312 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3cb26ad7-fe14-42ef-b550-50d980e72162 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32652025-07-15T21:17:38.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 284a80b6-adb5-46b9-9c52-bea33c184321 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32662025-07-15T21:17:38.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 284a80b6-adb5-46b9-9c52-bea33c184321 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32672025-07-15T21:17:38.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2652 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 284a80b6-adb5-46b9-9c52-bea33c184321 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32682025-07-15T21:17:39.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = de2a1e01-c230-41f3-b916-a83d12f61dd6 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32692025-07-15T21:17:39.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = de2a1e01-c230-41f3-b916-a83d12f61dd6 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32702025-07-15T21:17:39.802ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = de2a1e01-c230-41f3-b916-a83d12f61dd6 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32712025-07-15T21:17:39.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 633 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 6ee873a9-9d3d-475c-9994-38551903113d response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
32722025-07-15T21:17:40.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1fdb96c3-2821-4169-a231-fb6161162e9c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32732025-07-15T21:17:40.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1fdb96c3-2821-4169-a231-fb6161162e9c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32742025-07-15T21:17:40.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2680 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1fdb96c3-2821-4169-a231-fb6161162e9c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32752025-07-15T21:17:41.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = eaff6212-835f-4fcf-b455-fddf33d9bb20 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32762025-07-15T21:17:41.834ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = eaff6212-835f-4fcf-b455-fddf33d9bb20 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32772025-07-15T21:17:41.836ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2717 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = eaff6212-835f-4fcf-b455-fddf33d9bb20 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32782025-07-15T21:17:41.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 138 reason = Timeout
32792025-07-15T21:17:41.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32802025-07-15T21:17:41.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32812025-07-15T21:17:41.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32822025-07-15T21:17:41.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
32832025-07-15T21:17:41.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32842025-07-15T21:17:41.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32852025-07-15T21:17:41.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
32862025-07-15T21:17:41.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.067653ms iteration = 138 status = {"chicken_switches_updated":false}
32872025-07-15T21:17:42.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9c27bc34-fb83-4702-9ef8-b4456b0ba1dc roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32882025-07-15T21:17:42.852ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9c27bc34-fb83-4702-9ef8-b4456b0ba1dc resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32892025-07-15T21:17:42.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9c27bc34-fb83-4702-9ef8-b4456b0ba1dc response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32902025-07-15T21:17:43.868ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e68fe092-6c3c-4b09-aed1-7f5f7f5f367f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32912025-07-15T21:17:43.868ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e68fe092-6c3c-4b09-aed1-7f5f7f5f367f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32922025-07-15T21:17:43.870ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2797 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e68fe092-6c3c-4b09-aed1-7f5f7f5f367f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32932025-07-15T21:17:44.884ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 362d7823-3bb6-4164-9a66-8c002f2eb9ed roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32942025-07-15T21:17:44.885ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 362d7823-3bb6-4164-9a66-8c002f2eb9ed resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32952025-07-15T21:17:44.886ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2075 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 362d7823-3bb6-4164-9a66-8c002f2eb9ed response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
32962025-07-15T21:17:44.967ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:60849
32972025-07-15T21:17:44.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60849 req_id = 4ffce3a7-f37b-43ef-9c09-76a8b437e218 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
32982025-07-15T21:17:44.968ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60849 req_id = 4ffce3a7-f37b-43ef-9c09-76a8b437e218 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
32992025-07-15T21:17:44.970ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2281 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60849 req_id = 4ffce3a7-f37b-43ef-9c09-76a8b437e218 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?sort_by=id_ascending
33002025-07-15T21:17:44.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60849 req_id = 54451cbc-184b-4f9e-981d-9607e9b645a8 roles = RoleSet { roles: {} } uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
33012025-07-15T21:17:44.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60849 req_id = 54451cbc-184b-4f9e-981d-9607e9b645a8 resource = Database result = Ok(()) uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
33022025-07-15T21:17:44.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60849 req_id = 54451cbc-184b-4f9e-981d-9607e9b645a8 response_code = 200 uri = /metrics/collectors/bf08f219-8402-4dbd-b274-8bb245b334b8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODBlMDMwZC1jNGYwLTRjMmItYTRkYy1mZGQyM2RlZGNlOWIifX0%3D
33032025-07-15T21:17:45.465ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2140 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8a47d61a-cdef-45b6-be7a-947df13d3ece uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2025-07-15T21:17:45.465192419Z }
33042025-07-15T21:17:45.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8a47d61a-cdef-45b6-be7a-947df13d3ece roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
33052025-07-15T21:17:45.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8a47d61a-cdef-45b6-be7a-947df13d3ece resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
33062025-07-15T21:17:45.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 12414 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8a47d61a-cdef-45b6-be7a-947df13d3ece response_code = 204 uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
33072025-07-15T21:17:46.091ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b14ff91f-454b-4aee-a81e-85aba93889e7 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
33082025-07-15T21:17:46.091ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b14ff91f-454b-4aee-a81e-85aba93889e7 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
33092025-07-15T21:17:46.093ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2305 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b14ff91f-454b-4aee-a81e-85aba93889e7 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
33102025-07-15T21:17:46.814ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
33112025-07-15T21:17:46.814ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
33122025-07-15T21:17:46.814ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
33132025-07-15T21:17:46.814ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
33142025-07-15T21:17:46.814ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33152025-07-15T21:17:46.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33162025-07-15T21:17:46.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
33172025-07-15T21:17:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 308270617 } })
33182025-07-15T21:17:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 308271697 } })
33192025-07-15T21:17:46.818ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
33202025-07-15T21:17:46.818ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
33212025-07-15T21:17:46.818ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.215613ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
33222025-07-15T21:17:46.860ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
33232025-07-15T21:17:46.860ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33242025-07-15T21:17:46.861ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f478cac8-642f-4f11-b439-19461aea5e43", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
33252025-07-15T21:17:46.861ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33262025-07-15T21:17:46.861ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
33272025-07-15T21:17:46.861ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33282025-07-15T21:17:46.861ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "90aceded-ff25-4521-a751-609e14da363b", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
33292025-07-15T21:17:46.861ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33302025-07-15T21:17:46.863ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33312025-07-15T21:17:46.863ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33322025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
33332025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
33342025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
33352025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33362025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
33372025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33382025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
33392025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
33402025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33412025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33422025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 139 reason = Timeout
33432025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33442025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
33452025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
33462025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
33472025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33482025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33492025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
33502025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33512025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
33522025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33532025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33542025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33552025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
33562025-07-15T21:17:46.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33572025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33582025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33592025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33602025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33612025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33622025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33632025-07-15T21:17:46.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
33642025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
33652025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
33662025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33672025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
33682025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
33692025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33702025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
33712025-07-15T21:17:46.887ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
33722025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33732025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
33742025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
33752025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
33762025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
33772025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33782025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33792025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
33802025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33812025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
33822025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33832025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
33842025-07-15T21:17:46.887ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 378092801 } })
33852025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
33862025-07-15T21:17:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33872025-07-15T21:17:46.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33882025-07-15T21:17:46.889ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33892025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33902025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33912025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33922025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33932025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33942025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33952025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33962025-07-15T21:17:46.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33972025-07-15T21:17:46.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33982025-07-15T21:17:46.897ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33992025-07-15T21:17:46.897ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 12.251887ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
34002025-07-15T21:17:46.898ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34012025-07-15T21:17:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34022025-07-15T21:17:46.900ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34032025-07-15T21:17:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34042025-07-15T21:17:46.901ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34052025-07-15T21:17:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34062025-07-15T21:17:46.901ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34072025-07-15T21:17:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34082025-07-15T21:17:46.902ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 15.391438ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
34092025-07-15T21:17:46.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
34102025-07-15T21:17:46.902ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
34112025-07-15T21:17:46.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
34122025-07-15T21:17:46.903ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34132025-07-15T21:17:46.903ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34142025-07-15T21:17:46.903ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
34152025-07-15T21:17:46.903ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34162025-07-15T21:17:46.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34172025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34182025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34192025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34202025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34212025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34222025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34232025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34242025-07-15T21:17:46.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34252025-07-15T21:17:46.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34262025-07-15T21:17:46.905ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 19.692319ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34272025-07-15T21:17:46.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
34282025-07-15T21:17:46.906ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34292025-07-15T21:17:46.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34302025-07-15T21:17:46.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
34312025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34322025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34332025-07-15T21:17:46.907ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1674, tv_nsec: 397475852 } })
34342025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34352025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34362025-07-15T21:17:46.907ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
34372025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
34382025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 22.384888ms iteration = 139 status = {"chicken_switches_updated":false}
34392025-07-15T21:17:46.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34402025-07-15T21:17:46.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34412025-07-15T21:17:46.908ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34422025-07-15T21:17:46.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34432025-07-15T21:17:46.909ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34442025-07-15T21:17:46.909ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
34452025-07-15T21:17:46.909ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34462025-07-15T21:17:46.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34472025-07-15T21:17:46.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34482025-07-15T21:17:46.910ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34492025-07-15T21:17:46.910ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34502025-07-15T21:17:46.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34512025-07-15T21:17:46.911ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34522025-07-15T21:17:46.913ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34532025-07-15T21:17:46.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34542025-07-15T21:17:46.914ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34552025-07-15T21:17:46.914ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
34562025-07-15T21:17:46.914ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 29.02845ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
34572025-07-15T21:17:46.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34582025-07-15T21:17:46.914ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34592025-07-15T21:17:46.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
34602025-07-15T21:17:46.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34612025-07-15T21:17:46.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34622025-07-15T21:17:46.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34632025-07-15T21:17:46.921ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34642025-07-15T21:17:46.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34652025-07-15T21:17:46.928ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34662025-07-15T21:17:46.931ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
34672025-07-15T21:17:46.931ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34682025-07-15T21:17:46.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34692025-07-15T21:17:46.934ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34702025-07-15T21:17:46.937ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
34712025-07-15T21:17:46.937ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34722025-07-15T21:17:46.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34732025-07-15T21:17:46.938ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
34742025-07-15T21:17:46.938ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.65205ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
34752025-07-15T21:17:46.944ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34762025-07-15T21:17:46.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34772025-07-15T21:17:46.946ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34782025-07-15T21:17:46.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34792025-07-15T21:17:46.962ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 75.007978ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
34802025-07-15T21:17:46.967ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34812025-07-15T21:17:46.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34822025-07-15T21:17:46.968ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 81.711442ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
34832025-07-15T21:17:46.969ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34842025-07-15T21:17:46.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34852025-07-15T21:17:46.971ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34862025-07-15T21:17:46.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34872025-07-15T21:17:46.972ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.394107ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
34882025-07-15T21:17:46.973ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 88.353583ms iteration = 27 status = {"physical_disks_added":0}
34892025-07-15T21:17:46.977ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34902025-07-15T21:17:46.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34912025-07-15T21:17:46.980ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34922025-07-15T21:17:46.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34932025-07-15T21:17:46.982ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34942025-07-15T21:17:46.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
34952025-07-15T21:17:46.985ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34962025-07-15T21:17:46.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34972025-07-15T21:17:46.987ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 100.825225ms iteration = 24 status = 0
34982025-07-15T21:17:47.002ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 116.651024ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
34992025-07-15T21:17:47.017ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
35002025-07-15T21:17:47.017ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 132.104744ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
35012025-07-15T21:17:47.051ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35022025-07-15T21:17:47.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35032025-07-15T21:17:47.054ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 239.082737ms iteration = 24 status = null
35042025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
35052025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c11aa01-f6ad-4beb-a8c2-7713060ca650", "content-length": "78", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
35062025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
35072025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
35082025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
35092025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 250.886453ms iteration = 24 status = {}
35102025-07-15T21:17:47.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
35112025-07-15T21:17:47.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
35122025-07-15T21:17:47.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
35132025-07-15T21:17:47.067ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
35142025-07-15T21:17:47.068ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
35152025-07-15T21:17:47.068ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
35162025-07-15T21:17:47.069ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
35172025-07-15T21:17:47.069ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
35182025-07-15T21:17:47.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
35192025-07-15T21:17:47.070ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
35202025-07-15T21:17:47.070ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
35212025-07-15T21:17:47.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
35222025-07-15T21:17:47.071ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
35232025-07-15T21:17:47.071ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
35242025-07-15T21:17:47.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
35252025-07-15T21:17:47.072ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
35262025-07-15T21:17:47.072ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
35272025-07-15T21:17:47.072ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "edfa6c1a-f722-440c-9b85-886056a0897b", "content-length": "81", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
35282025-07-15T21:17:47.101ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
35292025-07-15T21:17:47.101ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
35302025-07-15T21:17:47.101ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4779774-6cea-4365-ac04-cdb4fa985896", "content-length": "583", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
35312025-07-15T21:17:47.101ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35322025-07-15T21:17:47.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35332025-07-15T21:17:47.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dcbb16b7-0985-4a93-b093-d6a5c8047e2f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
35342025-07-15T21:17:47.109ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dcbb16b7-0985-4a93-b093-d6a5c8047e2f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
35352025-07-15T21:17:47.111ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
35362025-07-15T21:17:47.111ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35372025-07-15T21:17:47.111ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3128 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dcbb16b7-0985-4a93-b093-d6a5c8047e2f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
35382025-07-15T21:17:47.112ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8ee9750-b008-45f7-a10b-4355c2bd4833", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
35392025-07-15T21:17:47.112ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35402025-07-15T21:17:47.113ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
35412025-07-15T21:17:47.114ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
35422025-07-15T21:17:47.114ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
35432025-07-15T21:17:47.114ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5a3aedc-c417-437f-aa44-127dba41edae", "content-length": "81", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
35442025-07-15T21:17:47.116ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
35452025-07-15T21:17:47.116ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
35462025-07-15T21:17:47.117ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "7219e982-4650-4096-90b1-fcbab49a9b74", "content-length": "81", "date": "Tue, 15 Jul 2025 21:17:46 GMT"} })
35472025-07-15T21:17:47.118ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 231.742438ms iteration = 24 status = {"errors":[],"sps":[]}
35482025-07-15T21:17:47.153ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {906189cb-29f6-4e19-9009-154148f7bcd3: {10.151.2.175: {82f14aa6-45ec-4950-9dbb-f7a4c0df2320}, 10.151.2.176: {82f14aa6-45ec-4950-9dbb-f7a4c0df2320}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
35492025-07-15T21:17:47.153ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
35502025-07-15T21:17:47.153ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5e298309-8c00-44d2-aa8e-98a406646ee3", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
35512025-07-15T21:17:47.153ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35522025-07-15T21:17:47.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35532025-07-15T21:17:47.156ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35542025-07-15T21:17:47.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35552025-07-15T21:17:47.159ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35562025-07-15T21:17:47.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35572025-07-15T21:17:47.162ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35582025-07-15T21:17:47.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35592025-07-15T21:17:47.164ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35602025-07-15T21:17:47.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35612025-07-15T21:17:47.168ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35622025-07-15T21:17:47.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35632025-07-15T21:17:47.171ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
35642025-07-15T21:17:47.171ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
35652025-07-15T21:17:47.171ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
35662025-07-15T21:17:47.171ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c574d8ac-305e-427c-8305-f618d02efe58", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
35672025-07-15T21:17:47.173ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 286.35809ms iteration = 24 status = {}
35682025-07-15T21:17:47.200ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
35692025-07-15T21:17:47.200ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "566bde4c-663c-4c87-8cf0-ce8124646a6e", "content-length": "810", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
35702025-07-15T21:17:47.200ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(1875700) }} sled = sock
35712025-07-15T21:17:47.200ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
35722025-07-15T21:17:47.200ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
35732025-07-15T21:17:47.203ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 317.71007ms iteration = 24 status = {}
35742025-07-15T21:17:47.211ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0/state
35752025-07-15T21:17:47.309ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35762025-07-15T21:17:47.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35772025-07-15T21:17:47.326ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
35782025-07-15T21:17:47.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35792025-07-15T21:17:47.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35802025-07-15T21:17:47.338ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
35812025-07-15T21:17:47.338ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
35822025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
35832025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
35842025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35852025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35862025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35872025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
35882025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35892025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
35902025-07-15T21:17:47.339ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
35912025-07-15T21:17:47.339ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35922025-07-15T21:17:47.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35932025-07-15T21:17:47.342ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
35942025-07-15T21:17:47.342ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
35952025-07-15T21:17:47.482ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9bff4069-f659-47c2-8788-61c1f3cfb351", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
35962025-07-15T21:17:47.482ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
35972025-07-15T21:17:47.482ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
35982025-07-15T21:17:47.489ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "06730fd5-6c6a-4fb0-9de8-e22dd7e32a78", "content-length": "682", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
35992025-07-15T21:17:47.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36002025-07-15T21:17:47.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36012025-07-15T21:17:47.494ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36022025-07-15T21:17:47.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36032025-07-15T21:17:47.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36042025-07-15T21:17:47.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36052025-07-15T21:17:47.500ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36062025-07-15T21:17:47.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36072025-07-15T21:17:47.508ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 622.541606ms iteration = 24 status = {}
36082025-07-15T21:17:47.996ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2140 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2025-07-15T21:17:45.478119613Z }
36092025-07-15T21:17:47.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36102025-07-15T21:17:47.996ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36112025-07-15T21:17:48.012ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 vmm_needs_update = true
36122025-07-15T21:17:48.012ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36132025-07-15T21:17:48.013ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36142025-07-15T21:17:48.021ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36152025-07-15T21:17:48.021ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36162025-07-15T21:17:48.023ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36172025-07-15T21:17:48.023ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36182025-07-15T21:17:48.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36192025-07-15T21:17:48.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36202025-07-15T21:17:48.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36212025-07-15T21:17:48.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36222025-07-15T21:17:48.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36232025-07-15T21:17:48.029ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Database result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36242025-07-15T21:17:48.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36252025-07-15T21:17:48.034ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, lookup_type: ById(bcc93570-eccd-4826-b5ca-d9e35b9da5ed) }, key: 247a180a-1aec-4f46-92ba-f4e123be73aa, lookup_type: ById(247a180a-1aec-4f46-92ba-f4e123be73aa) }, key: f15eb431-8b3b-4843-a3bc-474c382777a1, lookup_type: ById(f15eb431-8b3b-4843-a3bc-474c382777a1) } result = Ok(()) uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36262025-07-15T21:17:48.040ZINFOnexus (dropshot_internal): starting update saga for f15eb431-8b3b-4843-a3bc-474c382777a1 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1603 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2025-07-15T21:17:45.478119613Z }
36272025-07-15T21:17:48.040ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 44034 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f response_code = 204 uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36282025-07-15T21:17:48.040ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f3ed284-122f-4049-af13-58f7614f1ec5", "content-length": "132", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
36292025-07-15T21:17:48.042ZDEBGnexus (dropshot_internal): preparing instance-update saga for f15eb431-8b3b-4843-a3bc-474c382777a1... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b4695124-5c2d-4c62-8431-2e26b5461e7f uri = /vmms/f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36302025-07-15T21:17:48.042ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update
36312025-07-15T21:17:48.042ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"f15eb431-8b3b-4843-a3bc-474c382777a1","lookup_type":{"ById":"f15eb431-8b3b-4843-a3bc-474c382777a1"},"parent":{"key":"247a180a-1aec-4f46-92ba-f4e123be73aa","lookup_type":{"ById":"247a180a-1aec-4f46-92ba-f4e123be73aa"},"parent":{"key":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","lookup_type":{"ById":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36322025-07-15T21:17:48.042ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update
36332025-07-15T21:17:48.043ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 sled_id = 12003c51-8cc1-4676-bb02-79c3a895e0eb state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2025-07-15T21:17:45.478119613Z }, migration_in: None, migration_out: None } vmm_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36342025-07-15T21:17:48.043ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2140 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 sled_id = 12003c51-8cc1-4676-bb02-79c3a895e0eb vmm_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2025-07-15T21:17:45.478119613Z }
36352025-07-15T21:17:48.043ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 roles = RoleSet { roles: {} } sled_id = 12003c51-8cc1-4676-bb02-79c3a895e0eb vmm_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36362025-07-15T21:17:48.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 resource = Database result = Ok(()) sled_id = 12003c51-8cc1-4676-bb02-79c3a895e0eb vmm_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0
36372025-07-15T21:17:48.092ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
36382025-07-15T21:17:48.094ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.207122731s iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"destroyed":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
36392025-07-15T21:17:48.096ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update
36402025-07-15T21:17:48.096ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36412025-07-15T21:17:48.096ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36422025-07-15T21:17:48.096ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36432025-07-15T21:17:48.105ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36442025-07-15T21:17:48.105ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36452025-07-15T21:17:48.123ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36462025-07-15T21:17:48.123ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36472025-07-15T21:17:48.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = bff33e81-7876-4220-a788-db5579bab774 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
36482025-07-15T21:17:48.126ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = bff33e81-7876-4220-a788-db5579bab774 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
36492025-07-15T21:17:48.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2385 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = bff33e81-7876-4220-a788-db5579bab774 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
36502025-07-15T21:17:48.131ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36512025-07-15T21:17:48.131ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("6360a5ee-6ae4-4776-a50d-c97268b759ab")) node_id = 0 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36522025-07-15T21:17:48.135ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36532025-07-15T21:17:48.135ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36542025-07-15T21:17:48.157ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update
36552025-07-15T21:17:48.157ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36562025-07-15T21:17:48.157ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36572025-07-15T21:17:48.159ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36582025-07-15T21:17:48.159ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36592025-07-15T21:17:48.161ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36602025-07-15T21:17:48.161ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36612025-07-15T21:17:48.162ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36622025-07-15T21:17:48.163ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36632025-07-15T21:17:48.164ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36642025-07-15T21:17:48.164ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36652025-07-15T21:17:48.166ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36662025-07-15T21:17:48.166ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36672025-07-15T21:17:48.167ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36682025-07-15T21:17:48.172ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, lookup_type: ById(bcc93570-eccd-4826-b5ca-d9e35b9da5ed) }, key: 247a180a-1aec-4f46-92ba-f4e123be73aa, lookup_type: ById(247a180a-1aec-4f46-92ba-f4e123be73aa) }, key: f15eb431-8b3b-4843-a3bc-474c382777a1, lookup_type: ById(f15eb431-8b3b-4843-a3bc-474c382777a1) } result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36692025-07-15T21:17:48.172ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance updater_id = 6360a5ee-6ae4-4776-a50d-c97268b759ab
36702025-07-15T21:17:48.172ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36712025-07-15T21:17:48.172ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36722025-07-15T21:17:48.188ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance updater_id = 6360a5ee-6ae4-4776-a50d-c97268b759ab
36732025-07-15T21:17:48.188ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36742025-07-15T21:17:48.189ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance
36752025-07-15T21:17:48.193ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 locked_gen = Generation(Generation(2)) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = LockInstance updater_id = 6360a5ee-6ae4-4776-a50d-c97268b759ab
36762025-07-15T21:17:48.196ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36772025-07-15T21:17:48.196ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("6360a5ee-6ae4-4776-a50d-c97268b759ab")}) node_id = 1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36782025-07-15T21:17:48.200ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36792025-07-15T21:17:48.200ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
36802025-07-15T21:17:48.208ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36812025-07-15T21:17:48.208ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36822025-07-15T21:17:48.210ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36832025-07-15T21:17:48.210ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36842025-07-15T21:17:48.211ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36852025-07-15T21:17:48.211ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36862025-07-15T21:17:48.212ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36872025-07-15T21:17:48.216ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, lookup_type: ById(bcc93570-eccd-4826-b5ca-d9e35b9da5ed) }, key: 247a180a-1aec-4f46-92ba-f4e123be73aa, lookup_type: ById(247a180a-1aec-4f46-92ba-f4e123be73aa) }, key: f15eb431-8b3b-4843-a3bc-474c382777a1, lookup_type: ById(f15eb431-8b3b-4843-a3bc-474c382777a1) } result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36882025-07-15T21:17:48.216ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36892025-07-15T21:17:48.216ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
36902025-07-15T21:17:48.232ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: f904fa64-13d6-4ba2-9e2f-88b9bd351ba0, time_created: 2025-07-15T21:17:27.343427Z, time_deleted: None, instance_id: f15eb431-8b3b-4843-a3bc-474c382777a1, sled_id: 12003c51-8cc1-4676-bb02-79c3a895e0eb, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-07-15T21:17:45.478119Z, gen: Generation(Generation(6)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-07-15T21:17:25.034380Z, gen: Generation(Generation(3)), propolis_id: Some(f904fa64-13d6-4ba2-9e2f-88b9bd351ba0), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-07-15T21:17:48.232830762Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
36912025-07-15T21:17:48.240ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
36922025-07-15T21:17:48.240ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"f15eb431-8b3b-4843-a3bc-474c382777a1","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"f904fa64-13d6-4ba2-9e2f-88b9bd351ba0"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"f15eb431-8b3b-4843-a3bc-474c382777a1","lookup_type":{"ById":"f15eb431-8b3b-4843-a3bc-474c382777a1"},"parent":{"key":"247a180a-1aec-4f46-92ba-f4e123be73aa","lookup_type":{"ById":"247a180a-1aec-4f46-92ba-f4e123be73aa"},"parent":{"key":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","lookup_type":{"ById":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"6360a5ee-6ae4-4776-a50d-c97268b759ab"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"247a180a-1aec-4f46-92ba-f4e123be73aa","ram_diff":1073741824},"destroy_active_vmm":"f904fa64-13d6-4ba2-9e2f-88b9bd351ba0","destroy_target_vmm":null,"network_config":"Delete","new_intent":null,"new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-07-15T21:17:48.232830762Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36932025-07-15T21:17:48.240ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
36942025-07-15T21:17:48.280ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
36952025-07-15T21:17:48.280ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36962025-07-15T21:17:48.280ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36972025-07-15T21:17:48.280ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = c36bacd7-1265-4279-a15c-d892165be028
36982025-07-15T21:17:48.284ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
36992025-07-15T21:17:48.284ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37002025-07-15T21:17:48.288ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37012025-07-15T21:17:48.288ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37022025-07-15T21:17:48.293ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37032025-07-15T21:17:48.293ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("4dd79abc-2fbf-411f-b275-2d3460aedae7")) node_id = 0 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37042025-07-15T21:17:48.298ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37052025-07-15T21:17:48.298ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37062025-07-15T21:17:48.303ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 parent_lock = UpdaterLock { updater_id: 6360a5ee-6ae4-4776-a50d-c97268b759ab, locked_gen: Generation(Generation(2)) } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37072025-07-15T21:17:48.303ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = BecomeUpdater
37082025-07-15T21:17:48.303ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = BecomeUpdater
37092025-07-15T21:17:48.315ZDEBGnexus (SagaExecutor): inherited lock from 6360a5ee-6ae4-4776-a50d-c97268b759ab to 4dd79abc-2fbf-411f-b275-2d3460aedae7 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 6360a5ee-6ae4-4776-a50d-c97268b759ab saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = BecomeUpdater updater_id = 4dd79abc-2fbf-411f-b275-2d3460aedae7
37102025-07-15T21:17:48.315ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:987 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37112025-07-15T21:17:48.315ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37122025-07-15T21:17:48.315ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("4dd79abc-2fbf-411f-b275-2d3460aedae7")}) node_id = 1 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37132025-07-15T21:17:48.318ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37142025-07-15T21:17:48.318ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37152025-07-15T21:17:48.323ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37162025-07-15T21:17:48.323ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37172025-07-15T21:17:48.328ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37182025-07-15T21:17:48.328ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37192025-07-15T21:17:48.334ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1040 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37202025-07-15T21:17:48.334ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1
37212025-07-15T21:17:48.337ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UpdateNetworkConfig
37222025-07-15T21:17:48.337ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UpdateNetworkConfig
37232025-07-15T21:17:48.339ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UpdateNetworkConfig
37242025-07-15T21:17:48.340ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UpdateNetworkConfig
37252025-07-15T21:17:48.342ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
37262025-07-15T21:17:48.342ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UpdateNetworkConfig
37272025-07-15T21:17:48.342ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UpdateNetworkConfig
37282025-07-15T21:17:48.344ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
37292025-07-15T21:17:48.344ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
37302025-07-15T21:17:48.344ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
37312025-07-15T21:17:48.345ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
37322025-07-15T21:17:48.346ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
37332025-07-15T21:17:48.346ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
37342025-07-15T21:17:48.347ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
37352025-07-15T21:17:48.350ZDEBGnexus: notifying dendrite of updates instance_id = Some(f15eb431-8b3b-4843-a3bc-474c382777a1 (instance)) switch = switch0
37362025-07-15T21:17:48.350ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1675, tv_nsec: 841423302 } })
37372025-07-15T21:17:48.351ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1186
37382025-07-15T21:17:48.381ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
37392025-07-15T21:17:48.381ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37402025-07-15T21:17:48.382ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e40a2dd-ea87-4e17-b1e2-ec858eb52653", "content-length": "26", "date": "Tue, 15 Jul 2025 21:17:47 GMT"} })
37412025-07-15T21:17:48.382ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37422025-07-15T21:17:48.382ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37432025-07-15T21:17:48.417ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37442025-07-15T21:17:48.417ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37452025-07-15T21:17:48.421ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37462025-07-15T21:17:48.421ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37472025-07-15T21:17:48.426ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
37482025-07-15T21:17:48.427ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
37492025-07-15T21:17:48.440ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1115 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-07-15T21:17:48.428047Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 247a180a-1aec-4f46-92ba-f4e123be73aa, time_modified: Some(2025-07-15T21:17:48.428047Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, time_modified: Some(2025-07-15T21:17:48.428047Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37502025-07-15T21:17:48.440ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37512025-07-15T21:17:48.440ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37522025-07-15T21:17:48.443ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37532025-07-15T21:17:48.443ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37542025-07-15T21:17:48.448ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1150 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37552025-07-15T21:17:48.448ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UnassignOximeterProducer
37562025-07-15T21:17:48.449ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UnassignOximeterProducer
37572025-07-15T21:17:48.453ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = bf08f219-8402-4dbd-b274-8bb245b334b8 producer_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37582025-07-15T21:17:48.453ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UnassignOximeterProducer
37592025-07-15T21:17:48.453ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = UnassignOximeterProducer
37602025-07-15T21:17:48.489ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37612025-07-15T21:17:48.489ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/f15eb431-8b3b-4843-a3bc-474c382777a1
37622025-07-15T21:17:48.504ZDEBGnexus (SagaExecutor): client response oximeter-collector = f15eb431-8b3b-4843-a3bc-474c382777a1 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/f15eb431-8b3b-4843-a3bc-474c382777a1", status: 204, headers: {"x-request-id": "ba8875e8-95a7-4600-8af4-ae35e242b969", "date": "Tue, 15 Jul 2025 21:17:48 GMT"} }) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37632025-07-15T21:17:48.504ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = bf08f219-8402-4dbd-b274-8bb245b334b8 producer_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37642025-07-15T21:17:48.506ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37652025-07-15T21:17:48.506ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37662025-07-15T21:17:48.510ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37672025-07-15T21:17:48.510ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37682025-07-15T21:17:48.516ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 lock = UpdaterLock { updater_id: 4dd79abc-2fbf-411f-b275-2d3460aedae7, locked_gen: Generation(Generation(3)) } new_intent = None new_runtime = InstanceRuntimeState { time_updated: 2025-07-15T21:17:48.232830762Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37692025-07-15T21:17:48.516ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = CommitInstanceUpdates
37702025-07-15T21:17:48.516ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = CommitInstanceUpdates
37712025-07-15T21:17:48.565ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1201 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 new_runtime = InstanceRuntimeState { time_updated: 2025-07-15T21:17:48.232830762Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
37722025-07-15T21:17:48.565ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37732025-07-15T21:17:48.565ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Signaled
37742025-07-15T21:17:48.565ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37752025-07-15T21:17:48.565ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
37762025-07-15T21:17:48.565ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Signaled
37772025-07-15T21:17:48.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37782025-07-15T21:17:48.565ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37792025-07-15T21:17:48.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37802025-07-15T21:17:48.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37812025-07-15T21:17:48.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37822025-07-15T21:17:48.569ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37832025-07-15T21:17:48.569ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37842025-07-15T21:17:48.569ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37852025-07-15T21:17:48.569ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37862025-07-15T21:17:48.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37872025-07-15T21:17:48.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37882025-07-15T21:17:48.572ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37892025-07-15T21:17:48.572ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("f15eb431-8b3b-4843-a3bc-474c382777a1"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("f904fa64-13d6-4ba2-9e2f-88b9bd351ba0")}) node_id = 7 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37902025-07-15T21:17:48.572ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37912025-07-15T21:17:48.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37922025-07-15T21:17:48.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37932025-07-15T21:17:48.576ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37942025-07-15T21:17:48.576ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37952025-07-15T21:17:48.579ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37962025-07-15T21:17:48.579ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37972025-07-15T21:17:48.582ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
37982025-07-15T21:17:48.582ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = c36bacd7-1265-4279-a15c-d892165be028
37992025-07-15T21:17:48.586ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
38002025-07-15T21:17:48.586ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ReleaseSledResources
38012025-07-15T21:17:48.586ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ReleaseSledResources
38022025-07-15T21:17:48.590ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38032025-07-15T21:17:48.590ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38042025-07-15T21:17:48.594ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38052025-07-15T21:17:48.594ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38062025-07-15T21:17:48.597ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 propolis_id = f904fa64-13d6-4ba2-9e2f-88b9bd351ba0 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
38072025-07-15T21:17:48.597ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = MarkVmmDeleted
38082025-07-15T21:17:48.597ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = MarkVmmDeleted
38092025-07-15T21:17:48.652ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38102025-07-15T21:17:48.652ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38112025-07-15T21:17:48.656ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38122025-07-15T21:17:48.656ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38132025-07-15T21:17:48.659ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38142025-07-15T21:17:48.659ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38152025-07-15T21:17:48.663ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38162025-07-15T21:17:48.663ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38172025-07-15T21:17:48.666ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38182025-07-15T21:17:48.666ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38192025-07-15T21:17:48.667ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38202025-07-15T21:17:48.668ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38212025-07-15T21:17:48.669ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38222025-07-15T21:17:48.669ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38232025-07-15T21:17:48.671ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38242025-07-15T21:17:48.674ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, lookup_type: ById(bcc93570-eccd-4826-b5ca-d9e35b9da5ed) }, key: 247a180a-1aec-4f46-92ba-f4e123be73aa, lookup_type: ById(247a180a-1aec-4f46-92ba-f4e123be73aa) }, key: f15eb431-8b3b-4843-a3bc-474c382777a1, lookup_type: ById(f15eb431-8b3b-4843-a3bc-474c382777a1) } result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38252025-07-15T21:17:48.674ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38262025-07-15T21:17:48.674ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_node = ChainSuccessorSaga
38272025-07-15T21:17:48.680ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 intended_state = destroyed karmic_state = WillReincarnate needs_reincarnation = false saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update
38282025-07-15T21:17:48.680ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38292025-07-15T21:17:48.680ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38302025-07-15T21:17:48.683ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38312025-07-15T21:17:48.683ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38322025-07-15T21:17:48.685ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
38332025-07-15T21:17:48.685ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38342025-07-15T21:17:48.686ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0a255ef-b7c2-4f68-97c7-8534c3a378ea", "content-length": "372", "date": "Tue, 15 Jul 2025 21:17:48 GMT"} })
38352025-07-15T21:17:48.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38362025-07-15T21:17:48.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38372025-07-15T21:17:48.686ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38382025-07-15T21:17:48.686ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = c36bacd7-1265-4279-a15c-d892165be028
38392025-07-15T21:17:48.690ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = c36bacd7-1265-4279-a15c-d892165be028 sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38402025-07-15T21:17:48.690ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = c36bacd7-1265-4279-a15c-d892165be028
38412025-07-15T21:17:48.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38422025-07-15T21:17:48.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38432025-07-15T21:17:48.699ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38442025-07-15T21:17:48.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38452025-07-15T21:17:48.700ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
38462025-07-15T21:17:48.700ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
38472025-07-15T21:17:48.700ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3be23a02-f65c-453d-b914-94ca1b77c40e", "date": "Tue, 15 Jul 2025 21:17:48 GMT"} })
38482025-07-15T21:17:48.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38492025-07-15T21:17:48.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38502025-07-15T21:17:48.701ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38512025-07-15T21:17:48.702ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38522025-07-15T21:17:48.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38532025-07-15T21:17:48.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38542025-07-15T21:17:48.702ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38552025-07-15T21:17:48.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38562025-07-15T21:17:48.704ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38572025-07-15T21:17:48.705ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38582025-07-15T21:17:48.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38592025-07-15T21:17:48.706ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38602025-07-15T21:17:48.706ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38612025-07-15T21:17:48.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38622025-07-15T21:17:48.708ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
38632025-07-15T21:17:48.708ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38642025-07-15T21:17:48.708ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c262d060-a585-47bb-854f-267f3e876f75", "date": "Tue, 15 Jul 2025 21:17:48 GMT"} })
38652025-07-15T21:17:48.711ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 145.706775ms iteration = 25 status = {}
38662025-07-15T21:17:48.734ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38672025-07-15T21:17:48.734ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = c36bacd7-1265-4279-a15c-d892165be028 saga_name = instance-update saga_result = SagaResult { saga_id: c36bacd7-1265-4279-a15c-d892165be028, saga_log: SagaLog { saga_id: c36bacd7-1265-4279-a15c-d892165be028, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("4dd79abc-2fbf-411f-b275-2d3460aedae7")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("4dd79abc-2fbf-411f-b275-2d3460aedae7")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("f15eb431-8b3b-4843-a3bc-474c382777a1"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("f904fa64-13d6-4ba2-9e2f-88b9bd351ba0")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("f15eb431-8b3b-4843-a3bc-474c382777a1"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("f904fa64-13d6-4ba2-9e2f-88b9bd351ba0")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("4dd79abc-2fbf-411f-b275-2d3460aedae7"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("4dd79abc-2fbf-411f-b275-2d3460aedae7")}} }) }
38682025-07-15T21:17:48.734ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = c36bacd7-1265-4279-a15c-d892165be028 instance_id = f15eb431-8b3b-4843-a3bc-474c382777a1 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update
38692025-07-15T21:17:48.734ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38702025-07-15T21:17:48.734ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
38712025-07-15T21:17:48.738ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38722025-07-15T21:17:48.738ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
38732025-07-15T21:17:48.740ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
38742025-07-15T21:17:48.740ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8061261c-8ff5-491f-8be4-421fd327729d", "content-length": "691", "date": "Tue, 15 Jul 2025 21:17:48 GMT"} })
38752025-07-15T21:17:48.740ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
38762025-07-15T21:17:48.740ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
38772025-07-15T21:17:48.740ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
38782025-07-15T21:17:48.741ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38792025-07-15T21:17:48.741ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
38802025-07-15T21:17:48.742ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 176.727067ms iteration = 25 status = {}
38812025-07-15T21:17:48.745ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38822025-07-15T21:17:48.745ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7
38832025-07-15T21:17:48.752ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update sec_id = 6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38842025-07-15T21:17:48.752ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7 saga_name = start-instance-update saga_result = SagaResult { saga_id: 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7, saga_log: SagaLog { saga_id: 1e6510ca-97c2-4ad8-8bd4-e4c5711834c7, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("6360a5ee-6ae4-4776-a50d-c97268b759ab")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("6360a5ee-6ae4-4776-a50d-c97268b759ab")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("6360a5ee-6ae4-4776-a50d-c97268b759ab"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("6360a5ee-6ae4-4776-a50d-c97268b759ab")}} }) }
38852025-07-15T21:17:49.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2381a8dc-f8b6-41b3-90f6-3ebcf670d950 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38862025-07-15T21:17:49.144ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2381a8dc-f8b6-41b3-90f6-3ebcf670d950 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38872025-07-15T21:17:49.145ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2381a8dc-f8b6-41b3-90f6-3ebcf670d950 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38882025-07-15T21:17:49.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 679 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 235f7240-73cc-4805-bee1-876ab5f1dff9 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
38892025-07-15T21:17:50.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a48c0633-84be-4305-a9a2-d61ea1a52ccd roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38902025-07-15T21:17:50.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a48c0633-84be-4305-a9a2-d61ea1a52ccd resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38912025-07-15T21:17:50.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2360 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a48c0633-84be-4305-a9a2-d61ea1a52ccd response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38922025-07-15T21:17:51.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 60a2bbe0-1544-4658-91d2-575c0e457c7a roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38932025-07-15T21:17:51.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 60a2bbe0-1544-4658-91d2-575c0e457c7a resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38942025-07-15T21:17:51.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2502 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 60a2bbe0-1544-4658-91d2-575c0e457c7a response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
38952025-07-15T21:17:51.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 140 reason = Timeout
38962025-07-15T21:17:51.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
38972025-07-15T21:17:51.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
38982025-07-15T21:17:51.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38992025-07-15T21:17:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
39002025-07-15T21:17:51.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
39012025-07-15T21:17:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
39022025-07-15T21:17:51.891ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
39032025-07-15T21:17:51.891ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.19121ms iteration = 140 status = {"chicken_switches_updated":false}
39042025-07-15T21:17:52.196ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c76c2a36-312c-4d9f-961b-33b06bcced81 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39052025-07-15T21:17:52.196ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c76c2a36-312c-4d9f-961b-33b06bcced81 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39062025-07-15T21:17:52.198ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2278 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c76c2a36-312c-4d9f-961b-33b06bcced81 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39072025-07-15T21:17:53.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d6962a2c-f12e-4644-a386-8289aa7cfc68 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39082025-07-15T21:17:53.212ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d6962a2c-f12e-4644-a386-8289aa7cfc68 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39092025-07-15T21:17:53.214ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2546 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d6962a2c-f12e-4644-a386-8289aa7cfc68 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39102025-07-15T21:17:54.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 02add68f-992d-42ce-a5d7-6cb927afc25a roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39112025-07-15T21:17:54.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 02add68f-992d-42ce-a5d7-6cb927afc25a resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39122025-07-15T21:17:54.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2267 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 02add68f-992d-42ce-a5d7-6cb927afc25a response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39132025-07-15T21:17:55.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7e39b992-3876-4d60-b295-38da39a60f25 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39142025-07-15T21:17:55.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7e39b992-3876-4d60-b295-38da39a60f25 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39152025-07-15T21:17:55.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2791 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7e39b992-3876-4d60-b295-38da39a60f25 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39162025-07-15T21:17:56.268ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4a3d5645-35df-43a9-9229-e0078aff1872 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39172025-07-15T21:17:56.268ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4a3d5645-35df-43a9-9229-e0078aff1872 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39182025-07-15T21:17:56.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2781 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4a3d5645-35df-43a9-9229-e0078aff1872 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39192025-07-15T21:17:56.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
39202025-07-15T21:17:56.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39212025-07-15T21:17:56.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
39222025-07-15T21:17:56.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 141 reason = Timeout
39232025-07-15T21:17:56.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39242025-07-15T21:17:56.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
39252025-07-15T21:17:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39262025-07-15T21:17:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39272025-07-15T21:17:56.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
39282025-07-15T21:17:56.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39292025-07-15T21:17:56.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39302025-07-15T21:17:56.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39312025-07-15T21:17:56.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39322025-07-15T21:17:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39332025-07-15T21:17:56.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39342025-07-15T21:17:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39352025-07-15T21:17:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
39362025-07-15T21:17:56.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
39372025-07-15T21:17:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
39382025-07-15T21:17:56.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39392025-07-15T21:17:56.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39402025-07-15T21:17:56.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
39412025-07-15T21:17:56.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.441722ms iteration = 141 status = {"chicken_switches_updated":false}
39422025-07-15T21:17:56.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39432025-07-15T21:17:56.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
39442025-07-15T21:17:56.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39452025-07-15T21:17:56.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39462025-07-15T21:17:56.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
39472025-07-15T21:17:56.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.697647ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
39482025-07-15T21:17:56.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.254741ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39492025-07-15T21:17:57.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2a0c331d-763e-442c-a2a6-a1a5086c895a roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39502025-07-15T21:17:57.284ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2a0c331d-763e-442c-a2a6-a1a5086c895a resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39512025-07-15T21:17:57.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2270 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2a0c331d-763e-442c-a2a6-a1a5086c895a response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39522025-07-15T21:17:58.300ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4c315005-5fb3-49cb-a722-d8e3140f91f7 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39532025-07-15T21:17:58.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4c315005-5fb3-49cb-a722-d8e3140f91f7 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39542025-07-15T21:17:58.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2512 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4c315005-5fb3-49cb-a722-d8e3140f91f7 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39552025-07-15T21:17:59.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 34366cd2-46aa-4532-930c-dcd45632e6e1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39562025-07-15T21:17:59.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 34366cd2-46aa-4532-930c-dcd45632e6e1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39572025-07-15T21:17:59.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 34366cd2-46aa-4532-930c-dcd45632e6e1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39582025-07-15T21:17:59.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 687 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 816bde5c-9fcb-4e6b-b17b-83f737c4ce4b response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
39592025-07-15T21:18:00.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 65937f00-fdcc-4c24-abf1-ed5d221e3f52 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39602025-07-15T21:18:00.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 65937f00-fdcc-4c24-abf1-ed5d221e3f52 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39612025-07-15T21:18:00.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2426 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 65937f00-fdcc-4c24-abf1-ed5d221e3f52 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39622025-07-15T21:18:01.351ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b8ae67c0-b39a-4ca0-99f5-2976263cf4a2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39632025-07-15T21:18:01.351ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b8ae67c0-b39a-4ca0-99f5-2976263cf4a2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39642025-07-15T21:18:01.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2551 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b8ae67c0-b39a-4ca0-99f5-2976263cf4a2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39652025-07-15T21:18:01.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 142 reason = Timeout
39662025-07-15T21:18:01.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
39672025-07-15T21:18:01.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
39682025-07-15T21:18:01.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39692025-07-15T21:18:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
39702025-07-15T21:18:01.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
39712025-07-15T21:18:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
39722025-07-15T21:18:01.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
39732025-07-15T21:18:01.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.709394ms iteration = 142 status = {"chicken_switches_updated":false}
39742025-07-15T21:18:02.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f4113260-2f90-4066-9947-a26f833eb45c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39752025-07-15T21:18:02.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f4113260-2f90-4066-9947-a26f833eb45c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39762025-07-15T21:18:02.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2123 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f4113260-2f90-4066-9947-a26f833eb45c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39772025-07-15T21:18:03.382ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a7bfd7a7-5ea0-4c56-ac1d-bd39d1db6185 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39782025-07-15T21:18:03.383ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a7bfd7a7-5ea0-4c56-ac1d-bd39d1db6185 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39792025-07-15T21:18:03.385ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2896 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = a7bfd7a7-5ea0-4c56-ac1d-bd39d1db6185 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39802025-07-15T21:18:04.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 21c5eddc-a6a2-4f22-b93b-2daa51d831b8 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39812025-07-15T21:18:04.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 21c5eddc-a6a2-4f22-b93b-2daa51d831b8 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39822025-07-15T21:18:04.402ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2569 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 21c5eddc-a6a2-4f22-b93b-2daa51d831b8 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39832025-07-15T21:18:05.415ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9d4111a7-0611-435d-96f1-721b0e2da83b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39842025-07-15T21:18:05.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9d4111a7-0611-435d-96f1-721b0e2da83b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39852025-07-15T21:18:05.417ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2339 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9d4111a7-0611-435d-96f1-721b0e2da83b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39862025-07-15T21:18:06.431ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9b5360c4-db1c-437a-bfa6-48370864541c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39872025-07-15T21:18:06.431ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9b5360c4-db1c-437a-bfa6-48370864541c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39882025-07-15T21:18:06.433ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2604 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9b5360c4-db1c-437a-bfa6-48370864541c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
39892025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
39902025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
39912025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39922025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39932025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 143 reason = Timeout
39942025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
39952025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39962025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39972025-07-15T21:18:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
39982025-07-15T21:18:06.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39992025-07-15T21:18:06.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40002025-07-15T21:18:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40012025-07-15T21:18:06.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40022025-07-15T21:18:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40032025-07-15T21:18:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40042025-07-15T21:18:06.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40052025-07-15T21:18:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40062025-07-15T21:18:06.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40072025-07-15T21:18:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40082025-07-15T21:18:06.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40092025-07-15T21:18:06.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.650768ms iteration = 143 status = {"chicken_switches_updated":false}
40102025-07-15T21:18:06.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40112025-07-15T21:18:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
40122025-07-15T21:18:06.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40132025-07-15T21:18:06.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40142025-07-15T21:18:06.904ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
40152025-07-15T21:18:06.904ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.656028ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
40162025-07-15T21:18:06.921ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40172025-07-15T21:18:06.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40182025-07-15T21:18:06.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.622403ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40192025-07-15T21:18:07.447ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = da06f80f-7a52-4571-a600-9c53cb6970cc roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40202025-07-15T21:18:07.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = da06f80f-7a52-4571-a600-9c53cb6970cc resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40212025-07-15T21:18:07.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2245 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = da06f80f-7a52-4571-a600-9c53cb6970cc response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40222025-07-15T21:18:08.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3e608a73-3b0f-488b-bee5-0d2130b2f6e4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40232025-07-15T21:18:08.463ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3e608a73-3b0f-488b-bee5-0d2130b2f6e4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40242025-07-15T21:18:08.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2832 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3e608a73-3b0f-488b-bee5-0d2130b2f6e4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40252025-07-15T21:18:09.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 003d6f75-d750-43df-a22f-fa07838c4f53 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40262025-07-15T21:18:09.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 003d6f75-d750-43df-a22f-fa07838c4f53 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40272025-07-15T21:18:09.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2161 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 003d6f75-d750-43df-a22f-fa07838c4f53 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40282025-07-15T21:18:09.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 601 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 6c21fdcc-991e-411e-a1de-2a0001616407 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
40292025-07-15T21:18:10.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2d9aa57b-85fa-4504-a522-6138ebf0111f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40302025-07-15T21:18:10.497ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2d9aa57b-85fa-4504-a522-6138ebf0111f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40312025-07-15T21:18:10.499ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2346 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2d9aa57b-85fa-4504-a522-6138ebf0111f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40322025-07-15T21:18:11.513ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d8bc7bde-d48a-47a0-a192-67e8e0132292 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40332025-07-15T21:18:11.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d8bc7bde-d48a-47a0-a192-67e8e0132292 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40342025-07-15T21:18:11.515ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2370 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d8bc7bde-d48a-47a0-a192-67e8e0132292 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40352025-07-15T21:18:11.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 144 reason = Timeout
40362025-07-15T21:18:11.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40372025-07-15T21:18:11.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40382025-07-15T21:18:11.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40392025-07-15T21:18:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40402025-07-15T21:18:11.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40412025-07-15T21:18:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40422025-07-15T21:18:11.891ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40432025-07-15T21:18:11.891ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.153309ms iteration = 144 status = {"chicken_switches_updated":false}
40442025-07-15T21:18:12.529ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5ec27bd-dac0-4cc8-b81e-964818165467 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40452025-07-15T21:18:12.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5ec27bd-dac0-4cc8-b81e-964818165467 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40462025-07-15T21:18:12.531ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1924 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5ec27bd-dac0-4cc8-b81e-964818165467 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40472025-07-15T21:18:13.544ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d6015b83-2fa6-44ed-8472-e766df3d95d1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40482025-07-15T21:18:13.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d6015b83-2fa6-44ed-8472-e766df3d95d1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40492025-07-15T21:18:13.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d6015b83-2fa6-44ed-8472-e766df3d95d1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40502025-07-15T21:18:14.561ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5cbbc5f-1253-427b-99ae-19e9ef12cda9 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40512025-07-15T21:18:14.562ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5cbbc5f-1253-427b-99ae-19e9ef12cda9 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40522025-07-15T21:18:14.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2386 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e5cbbc5f-1253-427b-99ae-19e9ef12cda9 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40532025-07-15T21:18:15.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fc6f33b7-cce3-4ccd-861a-4d67b75a7903 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40542025-07-15T21:18:15.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fc6f33b7-cce3-4ccd-861a-4d67b75a7903 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40552025-07-15T21:18:15.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2602 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = fc6f33b7-cce3-4ccd-861a-4d67b75a7903 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40562025-07-15T21:18:16.593ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 92e60e3c-4019-473d-b3b7-b241ba86658c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40572025-07-15T21:18:16.593ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 92e60e3c-4019-473d-b3b7-b241ba86658c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40582025-07-15T21:18:16.595ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 92e60e3c-4019-473d-b3b7-b241ba86658c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
40592025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
40602025-07-15T21:18:16.814ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225
40612025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 24.421\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225"}
40622025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
40632025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
40642025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40652025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
40662025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
40672025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40682025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
40692025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
40702025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
40712025-07-15T21:18:16.814ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-07-15 21:08:16.814602831 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
40722025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
40732025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
40742025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
40752025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
40762025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
40772025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
40782025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40792025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
40802025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
40812025-07-15T21:18:16.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40822025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
40832025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40842025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40852025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40862025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40872025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
40882025-07-15T21:18:16.815ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4fcf02c2-2876-4bd3-b6eb-3250110a4c40.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("77d2eb9f-f483-4c87-9935-5142541c93df.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 306086640 } })
40892025-07-15T21:18:16.815ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ada38bf-5331-48b0-9340-612e7eca7fae.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1fb7ee6e-8db0-4006-be77-f524ca0c97d3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("4efcc6b2-ee9f-484a-bfd5-3d95c655eb63.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 306133262 } })
40902025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
40912025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.193601ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
40922025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
40932025-07-15T21:18:16.815ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.281273ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
40942025-07-15T21:18:16.820ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
40952025-07-15T21:18:16.820ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40962025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40972025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40982025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40992025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41002025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41012025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41022025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41032025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41042025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Timeout
41052025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41062025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
41072025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
41082025-07-15T21:18:16.821ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
41092025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 15.26\u{b5}s iteration = 17 status = "Disabled"
41102025-07-15T21:18:16.821ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41112025-07-15T21:18:16.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41122025-07-15T21:18:16.823ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41132025-07-15T21:18:16.823ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 314742304 } })
41142025-07-15T21:18:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41152025-07-15T21:18:16.824ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41162025-07-15T21:18:16.824ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.736402ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41172025-07-15T21:18:16.824ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
41182025-07-15T21:18:16.824ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41192025-07-15T21:18:16.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41202025-07-15T21:18:16.826ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 317096565 } })
41212025-07-15T21:18:16.826ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41222025-07-15T21:18:16.826ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41232025-07-15T21:18:16.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41242025-07-15T21:18:16.826ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
41252025-07-15T21:18:16.826ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-07-15 21:08:16.814602831 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
41262025-07-15T21:18:16.826ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.304548ms iteration = 13 status = {"expiration":"2025-07-15T21:08:16.814602831Z","pruned":[]}
41272025-07-15T21:18:16.827ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41282025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
41292025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
41302025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
41312025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
41322025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41332025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41342025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
41352025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
41362025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41372025-07-15T21:18:16.829ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41382025-07-15T21:18:16.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41392025-07-15T21:18:16.830ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41402025-07-15T21:18:16.831ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41412025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
41422025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
41432025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41442025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
41452025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
41462025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41472025-07-15T21:18:16.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41482025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
41492025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
41502025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
41512025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
41522025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
41532025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41542025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
41552025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41562025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41572025-07-15T21:18:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41582025-07-15T21:18:16.834ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41592025-07-15T21:18:16.834ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
41602025-07-15T21:18:16.834ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
41612025-07-15T21:18:16.834ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
41622025-07-15T21:18:16.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41632025-07-15T21:18:16.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41642025-07-15T21:18:16.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41652025-07-15T21:18:16.836ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41662025-07-15T21:18:16.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41672025-07-15T21:18:16.837ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41682025-07-15T21:18:16.839ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41692025-07-15T21:18:16.839ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.33247ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
41702025-07-15T21:18:16.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41712025-07-15T21:18:16.839ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41722025-07-15T21:18:16.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41732025-07-15T21:18:16.840ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41742025-07-15T21:18:16.841ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
41752025-07-15T21:18:16.841ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 8.346575ms iteration = 13 status = {"by_rx":{},"error":null}
41762025-07-15T21:18:16.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
41772025-07-15T21:18:16.842ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41782025-07-15T21:18:16.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41792025-07-15T21:18:16.843ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
41802025-07-15T21:18:16.843ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41812025-07-15T21:18:16.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41822025-07-15T21:18:16.845ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41832025-07-15T21:18:16.847ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
41842025-07-15T21:18:16.848ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 27.377487ms iteration = 6 status = {"nsuccess":5}
41852025-07-15T21:18:16.848ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41862025-07-15T21:18:16.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
41872025-07-15T21:18:16.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41882025-07-15T21:18:16.849ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41892025-07-15T21:18:16.849ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41902025-07-15T21:18:16.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41912025-07-15T21:18:16.850ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 16.979969ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"162.0.0","globs_reprocessed":{},"no_receivers":[]}
41922025-07-15T21:18:16.850ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41932025-07-15T21:18:16.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41942025-07-15T21:18:16.852ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41952025-07-15T21:18:16.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41962025-07-15T21:18:16.877ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
41972025-07-15T21:18:16.877ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
41982025-07-15T21:18:16.877ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 48.591156ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
41992025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
42002025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
42012025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
42022025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
42032025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42042025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
42052025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42062025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42072025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
42082025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
42092025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
42102025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42112025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42122025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 145 reason = Timeout
42132025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42142025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
42152025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42162025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
42172025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42182025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
42192025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42202025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42212025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
42222025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
42232025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
42242025-07-15T21:18:16.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42252025-07-15T21:18:16.885ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
42262025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
42272025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
42282025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42292025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
42302025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42312025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
42322025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
42332025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42342025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42352025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
42362025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
42372025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
42382025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
42392025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
42402025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42412025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42422025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42432025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42442025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42452025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42462025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
42472025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
42482025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42492025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
42502025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
42512025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42522025-07-15T21:18:16.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
42532025-07-15T21:18:16.886ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 377279230 } })
42542025-07-15T21:18:16.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42552025-07-15T21:18:16.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42562025-07-15T21:18:16.888ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42572025-07-15T21:18:16.888ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
42582025-07-15T21:18:16.888ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42592025-07-15T21:18:16.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42602025-07-15T21:18:16.897ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42612025-07-15T21:18:16.897ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42622025-07-15T21:18:16.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42632025-07-15T21:18:16.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42642025-07-15T21:18:16.901ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42652025-07-15T21:18:16.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42662025-07-15T21:18:16.901ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42672025-07-15T21:18:16.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42682025-07-15T21:18:16.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42692025-07-15T21:18:16.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42702025-07-15T21:18:16.902ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
42712025-07-15T21:18:16.902ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
42722025-07-15T21:18:16.902ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.280834ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
42732025-07-15T21:18:16.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42742025-07-15T21:18:16.905ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42752025-07-15T21:18:16.905ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42762025-07-15T21:18:16.906ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 20.743076ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
42772025-07-15T21:18:16.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42782025-07-15T21:18:16.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42792025-07-15T21:18:16.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42802025-07-15T21:18:16.910ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42812025-07-15T21:18:16.910ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42822025-07-15T21:18:16.911ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42832025-07-15T21:18:16.911ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42842025-07-15T21:18:16.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
42852025-07-15T21:18:16.914ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42862025-07-15T21:18:16.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42872025-07-15T21:18:16.914ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42882025-07-15T21:18:16.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42892025-07-15T21:18:16.915ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42902025-07-15T21:18:16.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42912025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
42922025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
42932025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
42942025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
42952025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 0 version = 2
42962025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42972025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
42982025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42992025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43002025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43012025-07-15T21:18:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
43022025-07-15T21:18:16.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43032025-07-15T21:18:16.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43042025-07-15T21:18:16.917ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43052025-07-15T21:18:16.919ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43062025-07-15T21:18:16.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43072025-07-15T21:18:16.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43082025-07-15T21:18:16.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43092025-07-15T21:18:16.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43102025-07-15T21:18:16.920ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43112025-07-15T21:18:16.921ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43122025-07-15T21:18:16.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43132025-07-15T21:18:16.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.700749ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43142025-07-15T21:18:16.922ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43152025-07-15T21:18:16.923ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1704, tv_nsec: 413884596 } })
43162025-07-15T21:18:16.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43172025-07-15T21:18:16.923ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
43182025-07-15T21:18:16.923ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-07-15T21:06:12.596852Z, time_modified: 2025-07-15T21:06:12.596852Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, db_silo: Silo { identity: SiloIdentity { id: bcc93570-eccd-4826-b5ca-d9e35b9da5ed, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-07-15T21:06:19.189707Z, time_modified: 2025-07-15T21:06:19.189707Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
43192025-07-15T21:18:16.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43202025-07-15T21:18:16.923ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 109.212282ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]}},"default_endpoint":{"silo_id":"bcc93570-eccd-4826-b5ca-d9e35b9da5ed","tls_certs":["4f967b21402c3c3472cbfc90043912204ba8cebe7095397f443a4cadedfd8cc8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
43212025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43222025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43232025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43242025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43252025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43262025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43272025-07-15T21:18:16.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43282025-07-15T21:18:16.929ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 6ce5cfad-6cf9-4e40-b36b-73f246d7a02d dns_zone_name = oxide.test found_so_far = 4 version = 2
43292025-07-15T21:18:16.929ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
43302025-07-15T21:18:16.929ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-07-15 21:06:19.267830 UTC dns_group = external generation = 2
43312025-07-15T21:18:16.929ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 114.840917ms iteration = 13 status = {"generation":2}
43322025-07-15T21:18:16.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
43332025-07-15T21:18:16.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43342025-07-15T21:18:16.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43352025-07-15T21:18:16.940ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 54.752064ms iteration = 28 status = {"physical_disks_added":0}
43362025-07-15T21:18:16.946ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
43372025-07-15T21:18:16.946ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 60.838952ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
43382025-07-15T21:18:16.977ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43392025-07-15T21:18:16.980ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43402025-07-15T21:18:16.980ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43412025-07-15T21:18:16.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43422025-07-15T21:18:16.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43432025-07-15T21:18:16.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
43442025-07-15T21:18:16.981ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
43452025-07-15T21:18:16.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
43462025-07-15T21:18:16.983ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43472025-07-15T21:18:16.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43482025-07-15T21:18:16.984ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
43492025-07-15T21:18:16.984ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 98.353132ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
43502025-07-15T21:18:16.990ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 104.239774ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
43512025-07-15T21:18:16.992ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 106.223024ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
43522025-07-15T21:18:16.993ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43532025-07-15T21:18:16.993ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
43542025-07-15T21:18:16.993ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43552025-07-15T21:18:16.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43562025-07-15T21:18:16.993ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 108.014311ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
43572025-07-15T21:18:16.996ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
43582025-07-15T21:18:16.996ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 110.094515ms iteration = 145 status = {"chicken_switches_updated":false}
43592025-07-15T21:18:16.996ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43602025-07-15T21:18:16.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43612025-07-15T21:18:16.996ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43622025-07-15T21:18:16.998ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 112.63543ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
43632025-07-15T21:18:16.999ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43642025-07-15T21:18:17.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
43652025-07-15T21:18:17.000ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43662025-07-15T21:18:17.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43672025-07-15T21:18:17.001ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
43682025-07-15T21:18:17.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
43692025-07-15T21:18:17.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
43702025-07-15T21:18:17.002ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43712025-07-15T21:18:17.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43722025-07-15T21:18:17.003ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43732025-07-15T21:18:17.003ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
43742025-07-15T21:18:17.003ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 117.65773ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
43752025-07-15T21:18:17.004ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
43762025-07-15T21:18:17.004ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
43772025-07-15T21:18:17.004ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
43782025-07-15T21:18:17.004ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43792025-07-15T21:18:17.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43802025-07-15T21:18:17.005ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.678222ms iteration = 25 status = 0
43812025-07-15T21:18:17.006ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43822025-07-15T21:18:17.010ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
43832025-07-15T21:18:17.010ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43842025-07-15T21:18:17.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43852025-07-15T21:18:17.013ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
43862025-07-15T21:18:17.013ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43872025-07-15T21:18:17.013ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "51a2146e-3584-4261-8542-7645564d034d", "content-length": "26", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
43882025-07-15T21:18:17.013ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43892025-07-15T21:18:17.016ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43902025-07-15T21:18:17.021ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
43912025-07-15T21:18:17.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
43922025-07-15T21:18:17.022ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
43932025-07-15T21:18:17.022ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43942025-07-15T21:18:17.023ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "143ab09b-f814-4821-90bd-29ccf8753979", "content-length": "26", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
43952025-07-15T21:18:17.023ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43962025-07-15T21:18:17.024ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
43972025-07-15T21:18:17.024ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 195.196894ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
43982025-07-15T21:18:17.024ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43992025-07-15T21:18:17.041ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
44002025-07-15T21:18:17.041ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c1b46b1-574d-481d-bd1c-39c561bc8134", "content-length": "78", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
44012025-07-15T21:18:17.041ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
44022025-07-15T21:18:17.041ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
44032025-07-15T21:18:17.042ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
44042025-07-15T21:18:17.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
44052025-07-15T21:18:17.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
44062025-07-15T21:18:17.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
44072025-07-15T21:18:17.044ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
44082025-07-15T21:18:17.045ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
44092025-07-15T21:18:17.046ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
44102025-07-15T21:18:17.046ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
44112025-07-15T21:18:17.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
44122025-07-15T21:18:17.047ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 161.881353ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44132025-07-15T21:18:17.049ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
44142025-07-15T21:18:17.049ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
44152025-07-15T21:18:17.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
44162025-07-15T21:18:17.050ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
44172025-07-15T21:18:17.050ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
44182025-07-15T21:18:17.051ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "9774fe6d-a8b3-471e-86ff-760f1998d223", "content-length": "81", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
44192025-07-15T21:18:17.052ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
44202025-07-15T21:18:17.052ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
44212025-07-15T21:18:17.052ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6438a2c-d698-45ba-9d76-140ce4e229dd", "content-length": "81", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
44222025-07-15T21:18:17.052ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
44232025-07-15T21:18:17.055ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
44242025-07-15T21:18:17.055ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
44252025-07-15T21:18:17.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
44262025-07-15T21:18:17.057ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
44272025-07-15T21:18:17.057ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
44282025-07-15T21:18:17.057ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44292025-07-15T21:18:17.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44302025-07-15T21:18:17.058ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
44312025-07-15T21:18:17.058ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
44322025-07-15T21:18:17.058ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f0e8129-1358-4716-87f9-f4cdb8f74147", "content-length": "81", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
44332025-07-15T21:18:17.059ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44342025-07-15T21:18:17.060ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
44352025-07-15T21:18:17.060ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 174.771576ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
44362025-07-15T21:18:17.061ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 175.042263ms iteration = 25 status = {"errors":[],"sps":[]}
44372025-07-15T21:18:17.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
44382025-07-15T21:18:17.063ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44392025-07-15T21:18:17.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44402025-07-15T21:18:17.066ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e2ff83cf-6d25-42a5-9a95-87ce4d7be7c0 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
44412025-07-15T21:18:17.066ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
44422025-07-15T21:18:17.066ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-07-15 21:06:16.954295 UTC dns_group = internal generation = 1
44432025-07-15T21:18:17.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 251.541209ms iteration = 13 status = {"generation":1}
44442025-07-15T21:18:17.123ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
44452025-07-15T21:18:17.123ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44462025-07-15T21:18:17.123ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "167c4b25-5358-4f13-b04d-9833348fd39f", "content-length": "26", "date": "Tue, 15 Jul 2025 21:18:16 GMT"} })
44472025-07-15T21:18:17.124ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44482025-07-15T21:18:17.125ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
44492025-07-15T21:18:17.131ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
44502025-07-15T21:18:17.139ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44512025-07-15T21:18:17.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44522025-07-15T21:18:17.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44532025-07-15T21:18:17.143ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44542025-07-15T21:18:17.143ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44552025-07-15T21:18:17.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44562025-07-15T21:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44572025-07-15T21:18:17.147ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44582025-07-15T21:18:17.147ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44592025-07-15T21:18:17.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44602025-07-15T21:18:17.218ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
44612025-07-15T21:18:17.324ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 510.337836ms iteration = 25 status = {}
44622025-07-15T21:18:17.400ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
44632025-07-15T21:18:17.400ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44642025-07-15T21:18:17.400ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7a2ab47-6bc5-4fbb-afb2-1c5319287467", "content-length": "372", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} })
44652025-07-15T21:18:17.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44662025-07-15T21:18:17.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44672025-07-15T21:18:17.419ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
44682025-07-15T21:18:17.419ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
44692025-07-15T21:18:17.419ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "64afb7e2-f323-41dd-9303-3978c2302963", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} })
44702025-07-15T21:18:17.419ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44712025-07-15T21:18:17.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44722025-07-15T21:18:17.422ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44732025-07-15T21:18:17.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44742025-07-15T21:18:17.424ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
44752025-07-15T21:18:17.424ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "490f61cf-d2b2-498b-91a2-730cfdb5342d", "content-length": "691", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} })
44762025-07-15T21:18:17.424ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
44772025-07-15T21:18:17.424ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
44782025-07-15T21:18:17.424ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
44792025-07-15T21:18:17.426ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44802025-07-15T21:18:17.426ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 541.127012ms iteration = 26 status = {}
44812025-07-15T21:18:17.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44822025-07-15T21:18:17.429ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
44832025-07-15T21:18:17.429ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44842025-07-15T21:18:17.430ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2d92bb54-e0de-4c88-a10d-fca714d16150", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} })
44852025-07-15T21:18:17.432ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 546.620954ms iteration = 26 status = {}
44862025-07-15T21:18:17.485ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
44872025-07-15T21:18:17.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
44882025-07-15T21:18:17.488ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 673.719049ms iteration = 25 status = null
44892025-07-15T21:18:17.489ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
44902025-07-15T21:18:17.580ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
44912025-07-15T21:18:17.580ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "0c4ce82c-672c-4abd-afe5-98b3fb45b2e0", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
44922025-07-15T21:18:17.580ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
44932025-07-15T21:18:17.582ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "bc39c9fd-eb65-4717-9845-2964555b846c", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
44942025-07-15T21:18:17.582ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
44952025-07-15T21:18:17.584ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
44962025-07-15T21:18:17.585ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 770.427337ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
44972025-07-15T21:18:17.591ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44982025-07-15T21:18:17.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44992025-07-15T21:18:17.593ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
45002025-07-15T21:18:17.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45012025-07-15T21:18:17.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45022025-07-15T21:18:17.602ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
45032025-07-15T21:18:17.603ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "66d92ccb-14fb-47c4-a896-75129f1e01fe", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45042025-07-15T21:18:17.603ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45052025-07-15T21:18:17.605ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "6dff8ec5-b08b-4a97-b601-8aff6d7cf2e1", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45062025-07-15T21:18:17.605ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45072025-07-15T21:18:17.607ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "73a166d1-85c2-4858-b946-2deaf02e17ce", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45082025-07-15T21:18:17.607ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45092025-07-15T21:18:17.608ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 38ed00e1-b605-4c7c-a9e5-0d5584cdebd2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45102025-07-15T21:18:17.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 38ed00e1-b605-4c7c-a9e5-0d5584cdebd2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45112025-07-15T21:18:17.609ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
45122025-07-15T21:18:17.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 794.9051ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
45132025-07-15T21:18:17.610ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2213 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 38ed00e1-b605-4c7c-a9e5-0d5584cdebd2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45142025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
45152025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
45162025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
45172025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
45182025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45192025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45202025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45212025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45222025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45232025-07-15T21:18:17.611ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45242025-07-15T21:18:17.612ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
45252025-07-15T21:18:17.612ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45262025-07-15T21:18:17.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45272025-07-15T21:18:17.615ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
45282025-07-15T21:18:17.615ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
45292025-07-15T21:18:17.735ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8274b259-0813-493d-9e8a-3746c6178700", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} })
45302025-07-15T21:18:17.735ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
45312025-07-15T21:18:17.735ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
45322025-07-15T21:18:17.735ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e7cd9fd-24ae-41d0-ac62-b160803bd0c9", "content-length": "682", "date": "Tue, 15 Jul 2025 21:18:17 GMT"} })
45332025-07-15T21:18:17.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45342025-07-15T21:18:17.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45352025-07-15T21:18:17.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45362025-07-15T21:18:17.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45372025-07-15T21:18:17.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45382025-07-15T21:18:17.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45392025-07-15T21:18:17.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45402025-07-15T21:18:17.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45412025-07-15T21:18:17.752ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 867.243969ms iteration = 25 status = {}
45422025-07-15T21:18:18.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 58620f4c-2cbb-450e-98bd-76c27dcc950e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45432025-07-15T21:18:18.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 58620f4c-2cbb-450e-98bd-76c27dcc950e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45442025-07-15T21:18:18.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2718 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 58620f4c-2cbb-450e-98bd-76c27dcc950e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45452025-07-15T21:18:19.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cdadea69-7503-45b1-a6a8-1740788e9190 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45462025-07-15T21:18:19.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cdadea69-7503-45b1-a6a8-1740788e9190 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45472025-07-15T21:18:19.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2066 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cdadea69-7503-45b1-a6a8-1740788e9190 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45482025-07-15T21:18:19.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 538 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 7df53cb6-f55a-44da-9ab2-f746ef5afdb5 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
45492025-07-15T21:18:20.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9e4ae2aa-1996-4329-801b-624e27ea245f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45502025-07-15T21:18:20.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9e4ae2aa-1996-4329-801b-624e27ea245f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45512025-07-15T21:18:20.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2313 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9e4ae2aa-1996-4329-801b-624e27ea245f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45522025-07-15T21:18:21.675ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 30aee724-9060-4f8c-99f7-ebbf1e7fe9ef roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45532025-07-15T21:18:21.676ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 30aee724-9060-4f8c-99f7-ebbf1e7fe9ef resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45542025-07-15T21:18:21.678ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2931 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 30aee724-9060-4f8c-99f7-ebbf1e7fe9ef response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45552025-07-15T21:18:21.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 146 reason = Timeout
45562025-07-15T21:18:21.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45572025-07-15T21:18:21.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45582025-07-15T21:18:21.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45592025-07-15T21:18:21.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
45602025-07-15T21:18:21.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45612025-07-15T21:18:21.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45622025-07-15T21:18:21.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
45632025-07-15T21:18:21.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.447967ms iteration = 146 status = {"chicken_switches_updated":false}
45642025-07-15T21:18:22.691ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4a77be06-1e79-459b-b6e3-993474600bdb roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45652025-07-15T21:18:22.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4a77be06-1e79-459b-b6e3-993474600bdb resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45662025-07-15T21:18:22.693ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2326 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4a77be06-1e79-459b-b6e3-993474600bdb response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45672025-07-15T21:18:23.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6c0a5077-5fa7-4a4a-b712-89d0f863067c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45682025-07-15T21:18:23.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6c0a5077-5fa7-4a4a-b712-89d0f863067c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45692025-07-15T21:18:23.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2514 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6c0a5077-5fa7-4a4a-b712-89d0f863067c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45702025-07-15T21:18:24.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ee7bb0a5-f47b-4ae3-9c43-ba661334202f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45712025-07-15T21:18:24.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ee7bb0a5-f47b-4ae3-9c43-ba661334202f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45722025-07-15T21:18:24.725ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2707 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ee7bb0a5-f47b-4ae3-9c43-ba661334202f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45732025-07-15T21:18:25.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 721cc44d-0e17-463a-8f3e-818bc914d48a roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45742025-07-15T21:18:25.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 721cc44d-0e17-463a-8f3e-818bc914d48a resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45752025-07-15T21:18:25.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2371 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 721cc44d-0e17-463a-8f3e-818bc914d48a response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45762025-07-15T21:18:26.755ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b2d21c8f-033d-44c3-a7aa-37395b37a03c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45772025-07-15T21:18:26.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b2d21c8f-033d-44c3-a7aa-37395b37a03c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45782025-07-15T21:18:26.757ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2296 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b2d21c8f-033d-44c3-a7aa-37395b37a03c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
45792025-07-15T21:18:26.884ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
45802025-07-15T21:18:26.884ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
45812025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45822025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 147 reason = Timeout
45832025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45842025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45852025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45862025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45872025-07-15T21:18:26.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45882025-07-15T21:18:26.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45892025-07-15T21:18:26.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45902025-07-15T21:18:26.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45912025-07-15T21:18:26.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45922025-07-15T21:18:26.889ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.303311ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45932025-07-15T21:18:26.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45942025-07-15T21:18:26.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45952025-07-15T21:18:26.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45962025-07-15T21:18:26.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
45972025-07-15T21:18:26.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45982025-07-15T21:18:26.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45992025-07-15T21:18:26.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46002025-07-15T21:18:26.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46012025-07-15T21:18:26.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46022025-07-15T21:18:26.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.776051ms iteration = 147 status = {"chicken_switches_updated":false}
46032025-07-15T21:18:26.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46042025-07-15T21:18:26.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
46052025-07-15T21:18:26.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46062025-07-15T21:18:26.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46072025-07-15T21:18:26.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
46082025-07-15T21:18:26.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.331292ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
46092025-07-15T21:18:27.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7f302927-c386-4db7-9868-69af2f6ff73e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46102025-07-15T21:18:27.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7f302927-c386-4db7-9868-69af2f6ff73e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46112025-07-15T21:18:27.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2594 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7f302927-c386-4db7-9868-69af2f6ff73e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46122025-07-15T21:18:28.789ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7634a00e-a400-4b02-ace8-0844dfb77749 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46132025-07-15T21:18:28.789ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7634a00e-a400-4b02-ace8-0844dfb77749 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46142025-07-15T21:18:28.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2524 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7634a00e-a400-4b02-ace8-0844dfb77749 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46152025-07-15T21:18:29.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 263cc00a-db0d-4e5b-95c0-6fb816d6deae roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46162025-07-15T21:18:29.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 263cc00a-db0d-4e5b-95c0-6fb816d6deae resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46172025-07-15T21:18:29.807ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 263cc00a-db0d-4e5b-95c0-6fb816d6deae response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46182025-07-15T21:18:29.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 630 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 42263cd1-16c8-4340-9cd4-7950e5522676 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
46192025-07-15T21:18:30.821ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2e600d54-ac84-4d3c-8cc4-06aace8ce42e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46202025-07-15T21:18:30.822ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2e600d54-ac84-4d3c-8cc4-06aace8ce42e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46212025-07-15T21:18:30.824ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2e600d54-ac84-4d3c-8cc4-06aace8ce42e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46222025-07-15T21:18:31.838ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cf598343-6f32-4265-bbca-41edf1c98e7b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46232025-07-15T21:18:31.839ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cf598343-6f32-4265-bbca-41edf1c98e7b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46242025-07-15T21:18:31.840ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2309 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = cf598343-6f32-4265-bbca-41edf1c98e7b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46252025-07-15T21:18:31.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 148 reason = Timeout
46262025-07-15T21:18:31.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46272025-07-15T21:18:31.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46282025-07-15T21:18:31.886ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46292025-07-15T21:18:31.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46302025-07-15T21:18:31.889ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46312025-07-15T21:18:31.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46322025-07-15T21:18:31.891ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46332025-07-15T21:18:31.891ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.277552ms iteration = 148 status = {"chicken_switches_updated":false}
46342025-07-15T21:18:32.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 837a1333-d2ee-4871-b9f9-2333d3bcfe96 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46352025-07-15T21:18:32.856ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 837a1333-d2ee-4871-b9f9-2333d3bcfe96 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46362025-07-15T21:18:32.858ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 837a1333-d2ee-4871-b9f9-2333d3bcfe96 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46372025-07-15T21:18:33.871ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 407f95f2-d16b-4f91-8241-149367512e26 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46382025-07-15T21:18:33.871ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 407f95f2-d16b-4f91-8241-149367512e26 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46392025-07-15T21:18:33.873ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2457 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 407f95f2-d16b-4f91-8241-149367512e26 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46402025-07-15T21:18:34.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 28659344-d01f-495e-983b-7eacac21c5f5 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46412025-07-15T21:18:34.888ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 28659344-d01f-495e-983b-7eacac21c5f5 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46422025-07-15T21:18:34.890ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2618 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 28659344-d01f-495e-983b-7eacac21c5f5 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46432025-07-15T21:18:35.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4dabdbc9-1387-463f-b8d3-bc7eff6656fb roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46442025-07-15T21:18:35.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4dabdbc9-1387-463f-b8d3-bc7eff6656fb resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46452025-07-15T21:18:35.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4dabdbc9-1387-463f-b8d3-bc7eff6656fb response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46462025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
46472025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46482025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
46492025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46502025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 149 reason = Timeout
46512025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46522025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46532025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46542025-07-15T21:18:36.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46552025-07-15T21:18:36.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46562025-07-15T21:18:36.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46572025-07-15T21:18:36.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46582025-07-15T21:18:36.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46592025-07-15T21:18:36.888ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.904571ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46602025-07-15T21:18:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46612025-07-15T21:18:36.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46622025-07-15T21:18:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46632025-07-15T21:18:36.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46642025-07-15T21:18:36.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46652025-07-15T21:18:36.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46662025-07-15T21:18:36.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46672025-07-15T21:18:36.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46682025-07-15T21:18:36.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46692025-07-15T21:18:36.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.667027ms iteration = 149 status = {"chicken_switches_updated":false}
46702025-07-15T21:18:36.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46712025-07-15T21:18:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
46722025-07-15T21:18:36.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46732025-07-15T21:18:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46742025-07-15T21:18:36.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
46752025-07-15T21:18:36.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.182787ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
46762025-07-15T21:18:36.921ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9443d067-a70f-4ec3-887b-bfd72dd08705 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46772025-07-15T21:18:36.921ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9443d067-a70f-4ec3-887b-bfd72dd08705 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46782025-07-15T21:18:36.923ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 9443d067-a70f-4ec3-887b-bfd72dd08705 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46792025-07-15T21:18:37.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 84e098aa-300e-468c-8571-31b0c1bceee3 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46802025-07-15T21:18:37.937ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 84e098aa-300e-468c-8571-31b0c1bceee3 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46812025-07-15T21:18:37.939ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2296 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 84e098aa-300e-468c-8571-31b0c1bceee3 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46822025-07-15T21:18:38.954ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1c635d08-743f-439a-9598-1e5f000b8bad roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46832025-07-15T21:18:38.954ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1c635d08-743f-439a-9598-1e5f000b8bad resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46842025-07-15T21:18:38.956ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2839 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1c635d08-743f-439a-9598-1e5f000b8bad response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46852025-07-15T21:18:39.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 58a2d902-38dd-443f-a118-a64412e63d21 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46862025-07-15T21:18:39.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 58a2d902-38dd-443f-a118-a64412e63d21 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46872025-07-15T21:18:39.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2283 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 58a2d902-38dd-443f-a118-a64412e63d21 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46882025-07-15T21:18:39.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 619 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = a4f03db1-873e-48f4-afd6-6594bda234bf response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
46892025-07-15T21:18:40.987ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0c4da215-0fdb-4afa-94f7-75a6191a7ce1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46902025-07-15T21:18:40.987ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0c4da215-0fdb-4afa-94f7-75a6191a7ce1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46912025-07-15T21:18:40.989ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 0c4da215-0fdb-4afa-94f7-75a6191a7ce1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
46922025-07-15T21:18:41.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 150 reason = Timeout
46932025-07-15T21:18:41.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46942025-07-15T21:18:41.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46952025-07-15T21:18:41.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46962025-07-15T21:18:41.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46972025-07-15T21:18:41.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46982025-07-15T21:18:41.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46992025-07-15T21:18:41.891ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
47002025-07-15T21:18:41.891ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.460907ms iteration = 150 status = {"chicken_switches_updated":false}
47012025-07-15T21:18:42.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e992fa5-4a50-4522-ae7a-e257f66bf5f1 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47022025-07-15T21:18:42.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e992fa5-4a50-4522-ae7a-e257f66bf5f1 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47032025-07-15T21:18:42.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1935 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 1e992fa5-4a50-4522-ae7a-e257f66bf5f1 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47042025-07-15T21:18:43.017ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b8a084b0-3cb0-46f9-8248-c0190cb285f4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47052025-07-15T21:18:43.017ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b8a084b0-3cb0-46f9-8248-c0190cb285f4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47062025-07-15T21:18:43.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2160 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = b8a084b0-3cb0-46f9-8248-c0190cb285f4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47072025-07-15T21:18:44.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5317171d-e426-4e2a-88e8-19d48144f00b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47082025-07-15T21:18:44.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5317171d-e426-4e2a-88e8-19d48144f00b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47092025-07-15T21:18:44.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2752 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5317171d-e426-4e2a-88e8-19d48144f00b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47102025-07-15T21:18:45.050ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7cca60a1-457f-4f7e-85a1-a4652412508d roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47112025-07-15T21:18:45.050ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7cca60a1-457f-4f7e-85a1-a4652412508d resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47122025-07-15T21:18:45.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2201 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 7cca60a1-457f-4f7e-85a1-a4652412508d response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47132025-07-15T21:18:46.066ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5d69e90f-c026-4033-96d7-2ec4ca59907c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47142025-07-15T21:18:46.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5d69e90f-c026-4033-96d7-2ec4ca59907c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47152025-07-15T21:18:46.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2687 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5d69e90f-c026-4033-96d7-2ec4ca59907c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
47162025-07-15T21:18:46.814ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
47172025-07-15T21:18:46.814ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
47182025-07-15T21:18:46.814ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
47192025-07-15T21:18:46.814ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
47202025-07-15T21:18:46.814ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
47212025-07-15T21:18:46.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
47222025-07-15T21:18:46.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
47232025-07-15T21:18:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 308040751 } })
47242025-07-15T21:18:46.817ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1186
47252025-07-15T21:18:46.817ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 308504363 } })
47262025-07-15T21:18:46.817ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.226373ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
47272025-07-15T21:18:46.817ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1186
47282025-07-15T21:18:46.854ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
47292025-07-15T21:18:46.854ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47302025-07-15T21:18:46.854ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a69d5ac-4d97-48e6-b084-0875f69b2ee4", "content-length": "26", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
47312025-07-15T21:18:46.854ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47322025-07-15T21:18:46.856ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47332025-07-15T21:18:46.858ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1194 zone_address = fd00:1122:3344:101::2
47342025-07-15T21:18:46.858ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47352025-07-15T21:18:46.859ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "13a7ff82-c5ca-4c5b-bbe3-37e5f2a31984", "content-length": "26", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
47362025-07-15T21:18:46.859ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1197 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47372025-07-15T21:18:46.861ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47382025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
47392025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
47402025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 151 reason = Timeout
47412025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
47422025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47432025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
47442025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
47452025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47462025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
47472025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47482025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
47492025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
47502025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47512025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47522025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
47532025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47542025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
47552025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47562025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
47572025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
47582025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
47592025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
47602025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
47612025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47622025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
47632025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
47642025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47652025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
47662025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47672025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
47682025-07-15T21:18:46.885ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
47692025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47702025-07-15T21:18:46.885ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
47712025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47722025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
47732025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
47742025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47752025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47762025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
47772025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47782025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47792025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47802025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47812025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
47822025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47832025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47842025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47852025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
47862025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
47872025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47882025-07-15T21:18:46.886ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 377444394 } })
47892025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47902025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47912025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47922025-07-15T21:18:46.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47932025-07-15T21:18:46.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47942025-07-15T21:18:46.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47952025-07-15T21:18:46.888ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47962025-07-15T21:18:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47972025-07-15T21:18:46.890ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47982025-07-15T21:18:46.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47992025-07-15T21:18:46.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48002025-07-15T21:18:46.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48012025-07-15T21:18:46.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48022025-07-15T21:18:46.893ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48032025-07-15T21:18:46.893ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.72641ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
48042025-07-15T21:18:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48052025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48062025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.5165ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
48072025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48082025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
48092025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48102025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48112025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48122025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48132025-07-15T21:18:46.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48142025-07-15T21:18:46.895ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.436764ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
48152025-07-15T21:18:46.895ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48162025-07-15T21:18:46.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
48172025-07-15T21:18:46.896ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
48182025-07-15T21:18:46.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
48192025-07-15T21:18:46.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48202025-07-15T21:18:46.896ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48212025-07-15T21:18:46.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48222025-07-15T21:18:46.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
48232025-07-15T21:18:46.897ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48242025-07-15T21:18:46.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48252025-07-15T21:18:46.897ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 12.145834ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48262025-07-15T21:18:46.898ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48272025-07-15T21:18:46.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48282025-07-15T21:18:46.899ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 13.785976ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
48292025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48302025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48312025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48322025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48332025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48342025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48352025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48362025-07-15T21:18:46.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48372025-07-15T21:18:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
48382025-07-15T21:18:46.901ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48392025-07-15T21:18:46.901ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
48402025-07-15T21:18:46.901ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 15.653714ms iteration = 151 status = {"chicken_switches_updated":false}
48412025-07-15T21:18:46.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48422025-07-15T21:18:46.901ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-12003c51-8cc1-4676-bb02-79c3a895e0eb.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1734, tv_nsec: 392696988 } })
48432025-07-15T21:18:46.902ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1186 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
48442025-07-15T21:18:46.902ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 16.595319ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
48452025-07-15T21:18:46.904ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48462025-07-15T21:18:46.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48472025-07-15T21:18:46.905ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48482025-07-15T21:18:46.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48492025-07-15T21:18:46.956ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48502025-07-15T21:18:46.956ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
48512025-07-15T21:18:46.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48522025-07-15T21:18:46.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48532025-07-15T21:18:46.959ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 145.163361ms iteration = 26 status = {}
48542025-07-15T21:18:46.960ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48552025-07-15T21:18:46.960ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48562025-07-15T21:18:46.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48572025-07-15T21:18:46.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48582025-07-15T21:18:46.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48592025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48602025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48612025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48622025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48632025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48642025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48652025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
48662025-07-15T21:18:46.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
48672025-07-15T21:18:46.964ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48682025-07-15T21:18:46.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
48692025-07-15T21:18:46.965ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48702025-07-15T21:18:46.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48712025-07-15T21:18:46.967ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
48722025-07-15T21:18:46.967ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 81.191169ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
48732025-07-15T21:18:46.967ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 12003c51-8cc1-4676-bb02-79c3a895e0eb, lookup_type: ById(12003c51-8cc1-4676-bb02-79c3a895e0eb) } result = Ok(())
48742025-07-15T21:18:46.967ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48752025-07-15T21:18:46.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48762025-07-15T21:18:46.969ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 84.051032ms iteration = 26 status = 0
48772025-07-15T21:18:46.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48782025-07-15T21:18:46.972ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48792025-07-15T21:18:46.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
48802025-07-15T21:18:46.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48812025-07-15T21:18:46.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48822025-07-15T21:18:46.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48832025-07-15T21:18:46.975ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48842025-07-15T21:18:46.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48852025-07-15T21:18:47.003ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
48862025-07-15T21:18:47.003ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 117.520256ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
48872025-07-15T21:18:47.007ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48882025-07-15T21:18:47.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48892025-07-15T21:18:47.009ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 194.915627ms iteration = 26 status = null
48902025-07-15T21:18:47.010ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48912025-07-15T21:18:47.010ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
48922025-07-15T21:18:47.010ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48932025-07-15T21:18:47.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48942025-07-15T21:18:47.013ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1194 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 zone_address = fd00:1122:3344:101::2
48952025-07-15T21:18:47.013ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48962025-07-15T21:18:47.013ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5181d4f-e2f3-449c-ac21-8f8945f9c49e", "content-length": "26", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
48972025-07-15T21:18:47.013ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1197 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48982025-07-15T21:18:47.015ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1231 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
48992025-07-15T21:18:47.019ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 133.296175ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49002025-07-15T21:18:47.019ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49012025-07-15T21:18:47.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49022025-07-15T21:18:47.021ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:389
49032025-07-15T21:18:47.021ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 135.84937ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
49042025-07-15T21:18:47.039ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
49052025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c629be2-4fff-4bca-a656-f9f74ce9dbab", "content-length": "78", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
49062025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
49072025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
49082025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
49092025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
49102025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
49112025-07-15T21:18:47.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
49122025-07-15T21:18:47.042ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
49132025-07-15T21:18:47.042ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
49142025-07-15T21:18:47.042ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
49152025-07-15T21:18:47.044ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
49162025-07-15T21:18:47.044ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
49172025-07-15T21:18:47.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
49182025-07-15T21:18:47.045ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
49192025-07-15T21:18:47.045ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
49202025-07-15T21:18:47.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
49212025-07-15T21:18:47.045ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
49222025-07-15T21:18:47.045ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
49232025-07-15T21:18:47.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
49242025-07-15T21:18:47.046ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = sled start_ena = None
49252025-07-15T21:18:47.046ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
49262025-07-15T21:18:47.047ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
49272025-07-15T21:18:47.047ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49282025-07-15T21:18:47.047ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a116b88-3b65-48cb-a54c-3142177e69cd", "content-length": "81", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
49292025-07-15T21:18:47.047ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "28d35868-29da-41d7-92f8-b8b862eb0f94", "content-length": "372", "date": "Tue, 15 Jul 2025 21:18:47 GMT"} })
49302025-07-15T21:18:47.047ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49312025-07-15T21:18:47.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49322025-07-15T21:18:47.056ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49332025-07-15T21:18:47.056ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
49342025-07-15T21:18:47.056ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 170.905066ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
49352025-07-15T21:18:47.078ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
49362025-07-15T21:18:47.078ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
49372025-07-15T21:18:47.079ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fb98fe16-1e98-4891-8e26-951352010937", "date": "Tue, 15 Jul 2025 21:18:47 GMT"} })
49382025-07-15T21:18:47.079ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49392025-07-15T21:18:47.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49402025-07-15T21:18:47.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49412025-07-15T21:18:47.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49422025-07-15T21:18:47.084ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 05fdf85d-ad34-414f-b0fd-78981b22e214 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
49432025-07-15T21:18:47.084ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 05fdf85d-ad34-414f-b0fd-78981b22e214 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
49442025-07-15T21:18:47.085ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 0 sp_type = switch start_ena = None
49452025-07-15T21:18:47.085ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
49462025-07-15T21:18:47.085ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ade2b83-e9db-4185-ba76-097fe6450b85", "content-length": "81", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
49472025-07-15T21:18:47.086ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49482025-07-15T21:18:47.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49492025-07-15T21:18:47.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 05fdf85d-ad34-414f-b0fd-78981b22e214 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
49502025-07-15T21:18:47.087ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = None gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 00000000-0000-0000-0000-000000000000 (ereporter_restart) slot = 1 sp_type = sled start_ena = None
49512025-07-15T21:18:47.087ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000
49522025-07-15T21:18:47.087ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?limit=255&restart_id=00000000-0000-0000-0000-000000000000", status: 200, headers: {"content-type": "application/json", "x-request-id": "87760c44-3818-40ed-a112-0ce61a26bd07", "content-length": "81", "date": "Tue, 15 Jul 2025 21:18:46 GMT"} })
49532025-07-15T21:18:47.089ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 203.431317ms iteration = 26 status = {"errors":[],"sps":[]}
49542025-07-15T21:18:47.089ZINFOnexus (BackgroundTasks): VPC route manager sled 12003c51-8cc1-4676-bb02-79c3a895e0eb push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
49552025-07-15T21:18:47.089ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49562025-07-15T21:18:47.090ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f499251a-b7f6-4992-8718-34c1e7eb759f", "date": "Tue, 15 Jul 2025 21:18:47 GMT"} })
49572025-07-15T21:18:47.091ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 205.930351ms iteration = 27 status = {}
49582025-07-15T21:18:47.122ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 236.908431ms iteration = 29 status = {"physical_disks_added":0}
49592025-07-15T21:18:47.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49602025-07-15T21:18:47.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49612025-07-15T21:18:47.133ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1efc4545-0f08-4c9c-8c3e-a547e125d877, rack_id: 0db2e78b-2285-496a-afcd-92e2ebe72e65, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(44e36d9d-f9b3-439b-840c-aa96beccd36d) } rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
49622025-07-15T21:18:47.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49632025-07-15T21:18:47.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49642025-07-15T21:18:47.142ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49652025-07-15T21:18:47.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49662025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
49672025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 44e36d9d-f9b3-439b-840c-aa96beccd36d
49682025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
49692025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
49702025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49712025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49722025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49732025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49742025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49752025-07-15T21:18:47.143ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49762025-07-15T21:18:47.144ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65
49772025-07-15T21:18:47.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49782025-07-15T21:18:47.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49792025-07-15T21:18:47.144ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49802025-07-15T21:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49812025-07-15T21:18:47.146ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49822025-07-15T21:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49832025-07-15T21:18:47.146ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch_location = Switch0
49842025-07-15T21:18:47.146ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
49852025-07-15T21:18:47.147ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49862025-07-15T21:18:47.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49872025-07-15T21:18:47.149ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49882025-07-15T21:18:47.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49892025-07-15T21:18:47.182ZDEBGnexus (BackgroundTasks): client request SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
49902025-07-15T21:18:47.185ZDEBGnexus (BackgroundTasks): client response SledAgent = 12003c51-8cc1-4676-bb02-79c3a895e0eb background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1db9b934-f954-45c9-951a-4ebb55f941b8", "content-length": "691", "date": "Tue, 15 Jul 2025 21:18:47 GMT"} })
49912025-07-15T21:18:47.185ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
49922025-07-15T21:18:47.185ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
49932025-07-15T21:18:47.185ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
49942025-07-15T21:18:47.186ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 300.71503ms iteration = 27 status = {}
49952025-07-15T21:18:47.270ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c9b6b20d-a4cb-4510-9c0b-975e740a5bf5", "date": "Tue, 15 Jul 2025 21:18:47 GMT"} })
49962025-07-15T21:18:47.270ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 switch = Switch0
49972025-07-15T21:18:47.270ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
49982025-07-15T21:18:47.275ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0db2e78b-2285-496a-afcd-92e2ebe72e65 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c7a2819-e991-4092-a14d-a3c442a5db7f", "content-length": "682", "date": "Tue, 15 Jul 2025 21:18:47 GMT"} })
49992025-07-15T21:18:47.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50002025-07-15T21:18:47.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50012025-07-15T21:18:47.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50022025-07-15T21:18:47.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50032025-07-15T21:18:47.283ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50042025-07-15T21:18:47.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50052025-07-15T21:18:47.285ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50062025-07-15T21:18:47.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50072025-07-15T21:18:47.292ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 407.161561ms iteration = 26 status = {}
50082025-07-15T21:18:48.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2e27158e-a10f-4a25-b79e-ef38bea203dd roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50092025-07-15T21:18:48.102ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2e27158e-a10f-4a25-b79e-ef38bea203dd resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50102025-07-15T21:18:48.104ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2561 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 2e27158e-a10f-4a25-b79e-ef38bea203dd response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50112025-07-15T21:18:49.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f0901f8d-a126-4c13-876a-0ef4b5ae44ae roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50122025-07-15T21:18:49.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f0901f8d-a126-4c13-876a-0ef4b5ae44ae resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50132025-07-15T21:18:49.122ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3153 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f0901f8d-a126-4c13-876a-0ef4b5ae44ae response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50142025-07-15T21:18:49.975ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 552 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 89540aec-378b-4f23-ac43-ba78f2180de8 response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
50152025-07-15T21:18:50.136ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d2f39fd2-031c-4723-8d59-ce30ddffa029 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50162025-07-15T21:18:50.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d2f39fd2-031c-4723-8d59-ce30ddffa029 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50172025-07-15T21:18:50.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = d2f39fd2-031c-4723-8d59-ce30ddffa029 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50182025-07-15T21:18:51.152ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5322f2c9-ef36-4295-9f00-5b0c338a79f4 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50192025-07-15T21:18:51.152ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5322f2c9-ef36-4295-9f00-5b0c338a79f4 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50202025-07-15T21:18:51.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3024 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5322f2c9-ef36-4295-9f00-5b0c338a79f4 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50212025-07-15T21:18:51.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 152 reason = Timeout
50222025-07-15T21:18:51.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50232025-07-15T21:18:51.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50242025-07-15T21:18:51.888ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50252025-07-15T21:18:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
50262025-07-15T21:18:51.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50272025-07-15T21:18:51.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50282025-07-15T21:18:51.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
50292025-07-15T21:18:51.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.76689ms iteration = 152 status = {"chicken_switches_updated":false}
50302025-07-15T21:18:52.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba73ebff-a5d0-44d9-8009-22febfa4ab00 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50312025-07-15T21:18:52.170ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba73ebff-a5d0-44d9-8009-22febfa4ab00 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50322025-07-15T21:18:52.172ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2264 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = ba73ebff-a5d0-44d9-8009-22febfa4ab00 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50332025-07-15T21:18:53.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c8879b16-2ba4-46c1-8421-fa3c939a2cd0 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50342025-07-15T21:18:53.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c8879b16-2ba4-46c1-8421-fa3c939a2cd0 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50352025-07-15T21:18:53.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = c8879b16-2ba4-46c1-8421-fa3c939a2cd0 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50362025-07-15T21:18:54.202ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4c61ba8d-7620-44ef-835b-3285a08a1568 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50372025-07-15T21:18:54.202ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4c61ba8d-7620-44ef-835b-3285a08a1568 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50382025-07-15T21:18:54.207ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 5937 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 4c61ba8d-7620-44ef-835b-3285a08a1568 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50392025-07-15T21:18:55.222ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f79f9cbf-be21-463e-8ac0-89b9110b502c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50402025-07-15T21:18:55.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f79f9cbf-be21-463e-8ac0-89b9110b502c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50412025-07-15T21:18:55.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2484 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f79f9cbf-be21-463e-8ac0-89b9110b502c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50422025-07-15T21:18:56.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b72505a-1849-4597-9af7-4a1aaa98cda6 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50432025-07-15T21:18:56.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b72505a-1849-4597-9af7-4a1aaa98cda6 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50442025-07-15T21:18:56.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2090 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 8b72505a-1849-4597-9af7-4a1aaa98cda6 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50452025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
50462025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50472025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
50482025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 153 reason = Timeout
50492025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50502025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50512025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50522025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50532025-07-15T21:18:56.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50542025-07-15T21:18:56.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50552025-07-15T21:18:56.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50562025-07-15T21:18:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50572025-07-15T21:18:56.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50582025-07-15T21:18:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
50592025-07-15T21:18:56.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50602025-07-15T21:18:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50612025-07-15T21:18:56.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50622025-07-15T21:18:56.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50632025-07-15T21:18:56.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50642025-07-15T21:18:56.893ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
50652025-07-15T21:18:56.893ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.087648ms iteration = 153 status = {"chicken_switches_updated":false}
50662025-07-15T21:18:56.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50672025-07-15T21:18:56.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
50682025-07-15T21:18:56.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50692025-07-15T21:18:56.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50702025-07-15T21:18:56.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
50712025-07-15T21:18:56.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.29569ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
50722025-07-15T21:18:56.921ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50732025-07-15T21:18:56.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50742025-07-15T21:18:56.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.479969ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50752025-07-15T21:18:57.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f973680c-ffb5-4192-8bb5-4f92bda7377e roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50762025-07-15T21:18:57.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f973680c-ffb5-4192-8bb5-4f92bda7377e resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50772025-07-15T21:18:57.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1785 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f973680c-ffb5-4192-8bb5-4f92bda7377e response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50782025-07-15T21:18:58.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = caebf0f7-6e3f-4f4e-b6da-8d1f2754cce2 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50792025-07-15T21:18:58.271ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = caebf0f7-6e3f-4f4e-b6da-8d1f2754cce2 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50802025-07-15T21:18:58.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = caebf0f7-6e3f-4f4e-b6da-8d1f2754cce2 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50812025-07-15T21:18:59.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 453f961b-fe00-4765-9da5-39b105f8e6a7 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50822025-07-15T21:18:59.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 453f961b-fe00-4765-9da5-39b105f8e6a7 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50832025-07-15T21:18:59.289ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2442 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 453f961b-fe00-4765-9da5-39b105f8e6a7 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50842025-07-15T21:18:59.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 634 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = a7a0362f-b51a-4597-bb6f-f2728874c2bf response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
50852025-07-15T21:19:00.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f7083f8c-3851-495a-85be-50267117f4c6 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50862025-07-15T21:19:00.303ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f7083f8c-3851-495a-85be-50267117f4c6 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50872025-07-15T21:19:00.305ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2235 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f7083f8c-3851-495a-85be-50267117f4c6 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50882025-07-15T21:19:01.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dc7536bb-9715-48b2-b4ab-fbe0b7aa39bc roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50892025-07-15T21:19:01.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dc7536bb-9715-48b2-b4ab-fbe0b7aa39bc resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50902025-07-15T21:19:01.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = dc7536bb-9715-48b2-b4ab-fbe0b7aa39bc response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
50912025-07-15T21:19:01.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 154 reason = Timeout
50922025-07-15T21:19:01.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50932025-07-15T21:19:01.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50942025-07-15T21:19:01.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50952025-07-15T21:19:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
50962025-07-15T21:19:01.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50972025-07-15T21:19:01.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50982025-07-15T21:19:01.891ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
50992025-07-15T21:19:01.891ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.799716ms iteration = 154 status = {"chicken_switches_updated":false}
51002025-07-15T21:19:02.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f171b430-3c7f-47ea-9279-2b8391cdc839 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51012025-07-15T21:19:02.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f171b430-3c7f-47ea-9279-2b8391cdc839 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51022025-07-15T21:19:02.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2900 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = f171b430-3c7f-47ea-9279-2b8391cdc839 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51032025-07-15T21:19:03.351ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 840ade3a-21bc-4a97-a652-4e39390ec7de roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51042025-07-15T21:19:03.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 840ade3a-21bc-4a97-a652-4e39390ec7de resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51052025-07-15T21:19:03.354ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2465 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 840ade3a-21bc-4a97-a652-4e39390ec7de response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51062025-07-15T21:19:04.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6322fd02-f4ba-4f28-bdc0-148c46a8290a roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51072025-07-15T21:19:04.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6322fd02-f4ba-4f28-bdc0-148c46a8290a resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51082025-07-15T21:19:04.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2399 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 6322fd02-f4ba-4f28-bdc0-148c46a8290a response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51092025-07-15T21:19:05.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 323841a0-0388-4531-9caf-a76cd6dca6bc roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51102025-07-15T21:19:05.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 323841a0-0388-4531-9caf-a76cd6dca6bc resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51112025-07-15T21:19:05.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2628 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 323841a0-0388-4531-9caf-a76cd6dca6bc response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51122025-07-15T21:19:06.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 12cf5fe8-7681-4d4d-8ff4-120bd2317c45 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51132025-07-15T21:19:06.401ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 12cf5fe8-7681-4d4d-8ff4-120bd2317c45 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51142025-07-15T21:19:06.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2196 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 12cf5fe8-7681-4d4d-8ff4-120bd2317c45 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51152025-07-15T21:19:06.884ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
51162025-07-15T21:19:06.884ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
51172025-07-15T21:19:06.884ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 155 reason = Timeout
51182025-07-15T21:19:06.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51192025-07-15T21:19:06.884ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51202025-07-15T21:19:06.884ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51212025-07-15T21:19:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51222025-07-15T21:19:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51232025-07-15T21:19:06.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51242025-07-15T21:19:06.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51252025-07-15T21:19:06.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51262025-07-15T21:19:06.887ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51272025-07-15T21:19:06.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51282025-07-15T21:19:06.889ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.811485ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51292025-07-15T21:19:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51302025-07-15T21:19:06.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51312025-07-15T21:19:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51322025-07-15T21:19:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
51332025-07-15T21:19:06.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51342025-07-15T21:19:06.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51352025-07-15T21:19:06.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
51362025-07-15T21:19:06.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.472773ms iteration = 155 status = {"chicken_switches_updated":false}
51372025-07-15T21:19:06.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51382025-07-15T21:19:06.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51392025-07-15T21:19:06.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51402025-07-15T21:19:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 29ade3fa-0e4a-43b6-9faa-920a0c49b225, lookup_type: ById(29ade3fa-0e4a-43b6-9faa-920a0c49b225) } result = Ok(())
51412025-07-15T21:19:06.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51422025-07-15T21:19:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51432025-07-15T21:19:06.905ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 original_time_created = 2025-07-15 21:06:11.411183 UTC target_id = 29ade3fa-0e4a-43b6-9faa-920a0c49b225 time_created = 2025-07-15 21:06:11.411183 UTC
51442025-07-15T21:19:06.905ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.726195ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"29ade3fa-0e4a-43b6-9faa-920a0c49b225","time_created":"2025-07-15T21:06:11.411183Z"}
51452025-07-15T21:19:07.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3da00488-bc60-4e07-9179-4995abaf0349 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51462025-07-15T21:19:07.418ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3da00488-bc60-4e07-9179-4995abaf0349 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51472025-07-15T21:19:07.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2730 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 3da00488-bc60-4e07-9179-4995abaf0349 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51482025-07-15T21:19:08.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 17f056f1-1671-4c72-8851-0c4592e72fa3 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51492025-07-15T21:19:08.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 17f056f1-1671-4c72-8851-0c4592e72fa3 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51502025-07-15T21:19:08.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2118 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 17f056f1-1671-4c72-8851-0c4592e72fa3 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51512025-07-15T21:19:09.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = db9e0a2e-eb19-4d58-8eaf-0cee40d731cf roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51522025-07-15T21:19:09.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = db9e0a2e-eb19-4d58-8eaf-0cee40d731cf resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51532025-07-15T21:19:09.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3011 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = db9e0a2e-eb19-4d58-8eaf-0cee40d731cf response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51542025-07-15T21:19:09.976ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 505 local_addr = [fd00:1122:3344:101::b]:48459 method = GET remote_addr = [fd00:1122:3344:101::d]:48081 req_id = 2a951e83-f258-4635-9085-35bbe0aae34f response_code = 200 uri = /6e23f75f-139f-4fef-bc78-c2a2188cf2e9
51552025-07-15T21:19:10.466ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 72bdb8d8-d712-4fba-be3e-1a1503d6e151 roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51562025-07-15T21:19:10.467ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 72bdb8d8-d712-4fba-be3e-1a1503d6e151 resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51572025-07-15T21:19:10.469ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2252 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 72bdb8d8-d712-4fba-be3e-1a1503d6e151 response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51582025-07-15T21:19:11.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5c08e8c6-a386-4fe9-8663-4ecf856e8c0b roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51592025-07-15T21:19:11.484ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5c08e8c6-a386-4fe9-8663-4ecf856e8c0b resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51602025-07-15T21:19:11.486ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2639 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 5c08e8c6-a386-4fe9-8663-4ecf856e8c0b response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51612025-07-15T21:19:11.885ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 156 reason = Timeout
51622025-07-15T21:19:11.885ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51632025-07-15T21:19:11.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51642025-07-15T21:19:11.887ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51652025-07-15T21:19:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
51662025-07-15T21:19:11.890ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51672025-07-15T21:19:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51682025-07-15T21:19:11.892ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
51692025-07-15T21:19:11.892ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.895078ms iteration = 156 status = {"chicken_switches_updated":false}
51702025-07-15T21:19:12.499ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e85b1e0a-bcc5-437e-bc4b-c603a29d875f roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51712025-07-15T21:19:12.499ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e85b1e0a-bcc5-437e-bc4b-c603a29d875f resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51722025-07-15T21:19:12.501ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2846 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = e85b1e0a-bcc5-437e-bc4b-c603a29d875f response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51732025-07-15T21:19:13.515ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 872f7ec7-0b6c-4f64-b6a6-e9b5c8a6162c roles = RoleSet { roles: {} } uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51742025-07-15T21:19:13.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 872f7ec7-0b6c-4f64-b6a6-e9b5c8a6162c resource = Database result = Ok(()) uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb
51752025-07-15T21:19:13.518ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2651 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36810 req_id = 872f7ec7-0b6c-4f64-b6a6-e9b5c8a6162c response_code = 200 uri = /probes/12003c51-8cc1-4676-bb02-79c3a895e0eb