Max level shown:
12025-04-17T00:30:01.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = be864c80-364b-44cb-a55e-d781cb36b349 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22025-04-17T00:30:01.150ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = be864c80-364b-44cb-a55e-d781cb36b349 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
32025-04-17T00:30:01.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3026 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = be864c80-364b-44cb-a55e-d781cb36b349 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
42025-04-17T00:30:01.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 548 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = ee7021cc-5c7e-4dbc-99b7-8331b90087d9 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
52025-04-17T00:30:02.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 77eca882-6e0a-467d-b3f1-5ba7171c6197 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
62025-04-17T00:30:02.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 77eca882-6e0a-467d-b3f1-5ba7171c6197 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
72025-04-17T00:30:02.185ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2481 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 77eca882-6e0a-467d-b3f1-5ba7171c6197 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
82025-04-17T00:30:03.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6b49ed40-1766-4a32-952b-524bed3147e3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
92025-04-17T00:30:03.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6b49ed40-1766-4a32-952b-524bed3147e3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
102025-04-17T00:30:03.209ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1851 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6b49ed40-1766-4a32-952b-524bed3147e3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
112025-04-17T00:30:04.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bf14e8b9-ba9d-4df1-8fa1-c03243cdae6d roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
122025-04-17T00:30:04.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bf14e8b9-ba9d-4df1-8fa1-c03243cdae6d resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
132025-04-17T00:30:04.233ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2536 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bf14e8b9-ba9d-4df1-8fa1-c03243cdae6d response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
142025-04-17T00:30:05.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d22fd412-0329-4ff6-9962-ea9e9b432188 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
152025-04-17T00:30:05.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d22fd412-0329-4ff6-9962-ea9e9b432188 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
162025-04-17T00:30:05.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2707 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d22fd412-0329-4ff6-9962-ea9e9b432188 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
172025-04-17T00:30:06.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 38f2cdfc-0035-45c3-97f9-fdc453ef26ec roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
182025-04-17T00:30:06.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 38f2cdfc-0035-45c3-97f9-fdc453ef26ec resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
192025-04-17T00:30:06.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2713 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 38f2cdfc-0035-45c3-97f9-fdc453ef26ec response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
202025-04-17T00:30:07.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2bd32289-223b-4656-b5e5-32d59a9d6c96 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
212025-04-17T00:30:07.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2bd32289-223b-4656-b5e5-32d59a9d6c96 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
222025-04-17T00:30:07.300ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2044 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2bd32289-223b-4656-b5e5-32d59a9d6c96 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
232025-04-17T00:30:07.960ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
242025-04-17T00:30:07.960ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
252025-04-17T00:30:07.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
262025-04-17T00:30:07.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
272025-04-17T00:30:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
282025-04-17T00:30:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
292025-04-17T00:30:07.962ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
302025-04-17T00:30:07.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
312025-04-17T00:30:07.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
322025-04-17T00:30:07.964ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.335684ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
332025-04-17T00:30:07.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
342025-04-17T00:30:07.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
352025-04-17T00:30:07.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
362025-04-17T00:30:07.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
372025-04-17T00:30:07.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
382025-04-17T00:30:07.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
392025-04-17T00:30:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
402025-04-17T00:30:07.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
412025-04-17T00:30:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
422025-04-17T00:30:07.981ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
432025-04-17T00:30:07.981ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.029519ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
442025-04-17T00:30:08.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cc1cd358-9498-46aa-840f-4188d5092554 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
452025-04-17T00:30:08.321ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cc1cd358-9498-46aa-840f-4188d5092554 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
462025-04-17T00:30:08.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cc1cd358-9498-46aa-840f-4188d5092554 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
472025-04-17T00:30:09.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f012a769-bae8-47ec-93a1-bd9d5d4babf2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
482025-04-17T00:30:09.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f012a769-bae8-47ec-93a1-bd9d5d4babf2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
492025-04-17T00:30:09.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2065 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f012a769-bae8-47ec-93a1-bd9d5d4babf2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
502025-04-17T00:30:10.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 32325daf-a10c-4964-b490-0a0d31e660a9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
512025-04-17T00:30:10.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 32325daf-a10c-4964-b490-0a0d31e660a9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
522025-04-17T00:30:10.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2197 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 32325daf-a10c-4964-b490-0a0d31e660a9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
532025-04-17T00:30:11.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = df37ce9d-36cc-4ad8-8066-e9b650dcdcfb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
542025-04-17T00:30:11.387ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = df37ce9d-36cc-4ad8-8066-e9b650dcdcfb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
552025-04-17T00:30:11.389ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2289 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = df37ce9d-36cc-4ad8-8066-e9b650dcdcfb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
562025-04-17T00:30:11.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 493 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = bdb2b360-6b97-4dee-b856-b20ed1ef7a32 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
572025-04-17T00:30:12.408ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = edbdfa14-e328-447a-8c46-7f1954c9bc06 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
582025-04-17T00:30:12.408ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = edbdfa14-e328-447a-8c46-7f1954c9bc06 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
592025-04-17T00:30:12.410ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1715 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = edbdfa14-e328-447a-8c46-7f1954c9bc06 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
602025-04-17T00:30:13.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8d4904cc-6bc4-48e8-9f87-9c299d325e9e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
612025-04-17T00:30:13.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8d4904cc-6bc4-48e8-9f87-9c299d325e9e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
622025-04-17T00:30:13.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2397 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8d4904cc-6bc4-48e8-9f87-9c299d325e9e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
632025-04-17T00:30:14.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 75da3657-9334-47ea-974f-b184d39c0fa8 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
642025-04-17T00:30:14.450ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 75da3657-9334-47ea-974f-b184d39c0fa8 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
652025-04-17T00:30:14.452ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2600 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 75da3657-9334-47ea-974f-b184d39c0fa8 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
662025-04-17T00:30:15.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d7bae2dd-f420-449a-a55c-08c80e29cb41 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
672025-04-17T00:30:15.472ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d7bae2dd-f420-449a-a55c-08c80e29cb41 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
682025-04-17T00:30:15.474ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2734 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d7bae2dd-f420-449a-a55c-08c80e29cb41 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
692025-04-17T00:30:16.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e6e7e097-9a31-414d-8ce4-8223999e59d0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
702025-04-17T00:30:16.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e6e7e097-9a31-414d-8ce4-8223999e59d0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
712025-04-17T00:30:16.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2455 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e6e7e097-9a31-414d-8ce4-8223999e59d0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
722025-04-17T00:30:16.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = be92aaef-b60e-451a-8489-b2cb173d4e69 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
732025-04-17T00:30:16.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = be92aaef-b60e-451a-8489-b2cb173d4e69 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
742025-04-17T00:30:16.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3549 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = be92aaef-b60e-451a-8489-b2cb173d4e69 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
752025-04-17T00:30:16.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 69d4a4c0-dd25-4357-9c3f-52f94fbac8df roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
762025-04-17T00:30:16.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 69d4a4c0-dd25-4357-9c3f-52f94fbac8df resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
772025-04-17T00:30:16.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2146 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 69d4a4c0-dd25-4357-9c3f-52f94fbac8df response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
782025-04-17T00:30:17.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 017c6ff0-3cc8-4466-9277-6a73b1f5c52a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
792025-04-17T00:30:17.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 017c6ff0-3cc8-4466-9277-6a73b1f5c52a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
802025-04-17T00:30:17.521ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2436 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 017c6ff0-3cc8-4466-9277-6a73b1f5c52a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
812025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
822025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
832025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
842025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
852025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
862025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
872025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
882025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
892025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
902025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
912025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
922025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
932025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
942025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
952025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
962025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
972025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
982025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
992025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1002025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
1012025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1022025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
1032025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1042025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1052025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
1062025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
1072025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
1082025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1092025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1102025-04-17T00:30:17.961ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-17 00:20:17.961209492 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
1112025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
1122025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
1132025-04-17T00:30:17.961ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
1142025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1152025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
1162025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
1172025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
1182025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1192025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
1202025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1212025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
1222025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1232025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
1242025-04-17T00:30:17.961ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3
1252025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
1262025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.79\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3"}
1272025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
1282025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1292025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
1302025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
1312025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1322025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1332025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
1342025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1352025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
1362025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1372025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
1382025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
1392025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
1402025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1412025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1422025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
1432025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1442025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
1452025-04-17T00:30:17.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1462025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1472025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
1482025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
1492025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1502025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1512025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1522025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1532025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1542025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1552025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1562025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1572025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1582025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1592025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1602025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1612025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1622025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1632025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1642025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1652025-04-17T00:30:17.961ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99145431-02b3-4b6e-aa5d-968f96332449.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 632719918 } })
1662025-04-17T00:30:17.961ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b3e44f3-f539-46b4-bb67-d442c7ebd3cf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4488d95c-7f86-4ca6-ab72-2e345a6a57bb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 632705717 } })
1672025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1682025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1692025-04-17T00:30:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1702025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1712025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1722025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1732025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1742025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1752025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1762025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
1772025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1782025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1792025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1802025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1812025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1822025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.078517ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
1832025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1842025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1852025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
1862025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.337904ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
1872025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1882025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1892025-04-17T00:30:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1902025-04-17T00:30:17.963ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 634676547 } })
1912025-04-17T00:30:17.964ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
1922025-04-17T00:30:17.964ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 635530178 } })
1932025-04-17T00:30:17.965ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
1942025-04-17T00:30:17.965ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1952025-04-17T00:30:17.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1962025-04-17T00:30:17.965ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.165676ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1972025-04-17T00:30:17.966ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1982025-04-17T00:30:17.967ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1992025-04-17T00:30:17.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2002025-04-17T00:30:17.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2012025-04-17T00:30:17.969ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2022025-04-17T00:30:17.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2032025-04-17T00:30:17.978ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2042025-04-17T00:30:17.981ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2052025-04-17T00:30:17.981ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2062025-04-17T00:30:17.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2072025-04-17T00:30:17.986ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2082025-04-17T00:30:17.988ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2092025-04-17T00:30:17.988ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2102025-04-17T00:30:17.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2112025-04-17T00:30:17.992ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-17 00:20:17.961209492 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
2122025-04-17T00:30:17.992ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 31.299648ms iteration = 10 status = {"expiration":"2025-04-17T00:20:17.961209492Z","pruned":[]}
2132025-04-17T00:30:17.997ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2142025-04-17T00:30:17.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2152025-04-17T00:30:18.000ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2162025-04-17T00:30:18.001ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2172025-04-17T00:30:18.001ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 40.259665ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
2182025-04-17T00:30:18.004ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2192025-04-17T00:30:18.004ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2202025-04-17T00:30:18.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2212025-04-17T00:30:18.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2222025-04-17T00:30:18.005ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2232025-04-17T00:30:18.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2242025-04-17T00:30:18.011ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2252025-04-17T00:30:18.015ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2262025-04-17T00:30:18.015ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2272025-04-17T00:30:18.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2282025-04-17T00:30:18.016ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2292025-04-17T00:30:18.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2302025-04-17T00:30:18.019ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2312025-04-17T00:30:18.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2322025-04-17T00:30:18.019ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2332025-04-17T00:30:18.021ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342025-04-17T00:30:18.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2352025-04-17T00:30:18.022ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2362025-04-17T00:30:18.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2372025-04-17T00:30:18.024ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2382025-04-17T00:30:18.025ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 64.000902ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
2392025-04-17T00:30:18.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2402025-04-17T00:30:18.025ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2412025-04-17T00:30:18.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2422025-04-17T00:30:18.028ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2432025-04-17T00:30:18.028ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2442025-04-17T00:30:18.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2452025-04-17T00:30:18.030ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2462025-04-17T00:30:18.032ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2472025-04-17T00:30:18.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2482025-04-17T00:30:18.034ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2492025-04-17T00:30:18.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2502025-04-17T00:30:18.038ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2512025-04-17T00:30:18.038ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 77.511742ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
2522025-04-17T00:30:18.038ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2532025-04-17T00:30:18.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2542025-04-17T00:30:18.038ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2552025-04-17T00:30:18.038ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 77.81468ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2562025-04-17T00:30:18.041ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2572025-04-17T00:30:18.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2582025-04-17T00:30:18.042ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2592025-04-17T00:30:18.042ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2602025-04-17T00:30:18.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2612025-04-17T00:30:18.043ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2622025-04-17T00:30:18.047ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2632025-04-17T00:30:18.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2642025-04-17T00:30:18.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2652025-04-17T00:30:18.051ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2662025-04-17T00:30:18.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2672025-04-17T00:30:18.052ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 722806547 } })
2682025-04-17T00:30:18.052ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
2692025-04-17T00:30:18.054ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2702025-04-17T00:30:18.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2712025-04-17T00:30:18.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2722025-04-17T00:30:18.058ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2732025-04-17T00:30:18.058ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2742025-04-17T00:30:18.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2752025-04-17T00:30:18.061ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2762025-04-17T00:30:18.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2772025-04-17T00:30:18.063ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2782025-04-17T00:30:18.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2792025-04-17T00:30:18.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2802025-04-17T00:30:18.064ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2812025-04-17T00:30:18.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2822025-04-17T00:30:18.065ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2832025-04-17T00:30:18.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2842025-04-17T00:30:18.066ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2852025-04-17T00:30:18.066ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2862025-04-17T00:30:18.066ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 104.746928ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2872025-04-17T00:30:18.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2882025-04-17T00:30:18.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2892025-04-17T00:30:18.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2902025-04-17T00:30:18.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2912025-04-17T00:30:18.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2922025-04-17T00:30:18.069ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2932025-04-17T00:30:18.069ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2942025-04-17T00:30:18.069ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2952025-04-17T00:30:18.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2962025-04-17T00:30:18.069ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2972025-04-17T00:30:18.069ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2982025-04-17T00:30:18.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2992025-04-17T00:30:18.074ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 113.072318ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3002025-04-17T00:30:18.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012025-04-17T00:30:18.077ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 116.002651ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
3022025-04-17T00:30:18.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
3032025-04-17T00:30:18.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3042025-04-17T00:30:18.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3052025-04-17T00:30:18.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3062025-04-17T00:30:18.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3072025-04-17T00:30:18.086ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3082025-04-17T00:30:18.089ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 128.543998ms iteration = 21 status = {"physical_disks_added":0}
3092025-04-17T00:30:18.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
3102025-04-17T00:30:18.089ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3112025-04-17T00:30:18.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3122025-04-17T00:30:18.093ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 131.877602ms iteration = 5 status = {"nsuccess":5}
3132025-04-17T00:30:18.094ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 133.039691ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3142025-04-17T00:30:18.096ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3152025-04-17T00:30:18.096ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3162025-04-17T00:30:18.096ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3172025-04-17T00:30:18.096ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3182025-04-17T00:30:18.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3192025-04-17T00:30:18.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3202025-04-17T00:30:18.104ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
3212025-04-17T00:30:18.104ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 143.35213ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
3222025-04-17T00:30:18.107ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3232025-04-17T00:30:18.107ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3242025-04-17T00:30:18.107ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3252025-04-17T00:30:18.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3262025-04-17T00:30:18.114ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3272025-04-17T00:30:18.114ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3282025-04-17T00:30:18.114ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3292025-04-17T00:30:18.114ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3302025-04-17T00:30:18.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3312025-04-17T00:30:18.116ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3322025-04-17T00:30:18.116ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
3332025-04-17T00:30:18.116ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 154.863901ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3342025-04-17T00:30:18.116ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3352025-04-17T00:30:18.120ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3362025-04-17T00:30:18.120ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3372025-04-17T00:30:18.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3382025-04-17T00:30:18.124ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3392025-04-17T00:30:18.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3402025-04-17T00:30:18.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3412025-04-17T00:30:18.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3422025-04-17T00:30:18.129ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3432025-04-17T00:30:18.130ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3442025-04-17T00:30:18.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3452025-04-17T00:30:18.132ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3462025-04-17T00:30:18.132ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3472025-04-17T00:30:18.132ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3482025-04-17T00:30:18.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3492025-04-17T00:30:18.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
3502025-04-17T00:30:18.133ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3512025-04-17T00:30:18.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3522025-04-17T00:30:18.135ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 174.770753ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3532025-04-17T00:30:18.136ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3542025-04-17T00:30:18.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3552025-04-17T00:30:18.137ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 176.190648ms iteration = 19 status = 0
3562025-04-17T00:30:18.138ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572025-04-17T00:30:18.138ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 177.08449ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3582025-04-17T00:30:18.138ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3592025-04-17T00:30:18.140ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3602025-04-17T00:30:18.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3612025-04-17T00:30:18.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3622025-04-17T00:30:18.141ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3632025-04-17T00:30:18.141ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3642025-04-17T00:30:18.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3652025-04-17T00:30:18.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3662025-04-17T00:30:18.143ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3672025-04-17T00:30:18.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3682025-04-17T00:30:18.147ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3692025-04-17T00:30:18.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3702025-04-17T00:30:18.148ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
3712025-04-17T00:30:18.148ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3722025-04-17T00:30:18.149ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef279e2e-0013-4e5e-ae42-1b23af14f288", "content-length": "26", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
3732025-04-17T00:30:18.149ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3742025-04-17T00:30:18.151ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3752025-04-17T00:30:18.189ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3762025-04-17T00:30:18.189ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3772025-04-17T00:30:18.189ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3782025-04-17T00:30:18.190ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3792025-04-17T00:30:18.190ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 0 version = 2
3802025-04-17T00:30:18.190ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3812025-04-17T00:30:18.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3822025-04-17T00:30:18.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3832025-04-17T00:30:18.194ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3842025-04-17T00:30:18.194ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3852025-04-17T00:30:18.194ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 233.374938ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3862025-04-17T00:30:18.196ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3872025-04-17T00:30:18.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3882025-04-17T00:30:18.212ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-17T00:21:15.858908Z, time_modified: 2025-04-17T00:21:15.858908Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }) }
3892025-04-17T00:30:18.212ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 251.004342ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]}},"default_endpoint":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3902025-04-17T00:30:18.213ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
3912025-04-17T00:30:18.213ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 252.168532ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3922025-04-17T00:30:18.218ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
3932025-04-17T00:30:18.218ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3942025-04-17T00:30:18.218ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2f6c59a-fb14-4653-9038-19768e8dc5d4", "content-length": "26", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
3952025-04-17T00:30:18.218ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3962025-04-17T00:30:18.220ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3972025-04-17T00:30:18.263ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
3982025-04-17T00:30:18.263ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3992025-04-17T00:30:18.264ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "caec911b-d3bf-4625-bcf1-d8b029a81330", "content-length": "26", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
4002025-04-17T00:30:18.264ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4012025-04-17T00:30:18.270ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
4022025-04-17T00:30:18.295ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
4032025-04-17T00:30:18.295ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 333.699506ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4042025-04-17T00:30:18.335ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4052025-04-17T00:30:18.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4062025-04-17T00:30:18.335ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4072025-04-17T00:30:18.419ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 458.472168ms iteration = 19 status = {}
4082025-04-17T00:30:18.428ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4092025-04-17T00:30:18.428ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e0351caf-4e48-4dff-989d-b0ee7d0c1516", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4102025-04-17T00:30:18.428ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4112025-04-17T00:30:18.430ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2349ed7e-b64b-4fb3-a8dc-19e4a4e01981", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4122025-04-17T00:30:18.430ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4132025-04-17T00:30:18.431ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4142025-04-17T00:30:18.431ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 470.433519ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4152025-04-17T00:30:18.439ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4162025-04-17T00:30:18.439ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4172025-04-17T00:30:18.439ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "81da3dd5-761d-4301-80d6-0d755dcb7cc9", "content-length": "372", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
4182025-04-17T00:30:18.439ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4192025-04-17T00:30:18.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4202025-04-17T00:30:18.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4212025-04-17T00:30:18.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4222025-04-17T00:30:18.442ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4232025-04-17T00:30:18.445ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4242025-04-17T00:30:18.445ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4252025-04-17T00:30:18.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4262025-04-17T00:30:18.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4272025-04-17T00:30:18.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4282025-04-17T00:30:18.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4292025-04-17T00:30:18.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4302025-04-17T00:30:18.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4312025-04-17T00:30:18.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4322025-04-17T00:30:18.448ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 487.675723ms iteration = 19 status = null
4332025-04-17T00:30:18.450ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 1 version = 2
4342025-04-17T00:30:18.450ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4352025-04-17T00:30:18.450ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-17 00:21:19.348689 UTC dns_group = external generation = 2
4362025-04-17T00:30:18.450ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 489.480479ms iteration = 10 status = {"generation":2}
4372025-04-17T00:30:18.451ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
4382025-04-17T00:30:18.451ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4392025-04-17T00:30:18.451ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-17 00:21:18.231041 UTC dns_group = internal generation = 1
4402025-04-17T00:30:18.451ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 490.550956ms iteration = 10 status = {"generation":1}
4412025-04-17T00:30:18.456ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4422025-04-17T00:30:18.456ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "0eda5db8-628d-4234-b82d-d3705ca8df8e", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4432025-04-17T00:30:18.456ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4442025-04-17T00:30:18.459ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "17fd3d22-0f41-4943-aef5-1944cd56471e", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4452025-04-17T00:30:18.459ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4462025-04-17T00:30:18.460ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "cd4a14bf-fff1-4912-812e-0584abeb977b", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4472025-04-17T00:30:18.460ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4482025-04-17T00:30:18.462ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4492025-04-17T00:30:18.462ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 501.01417ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4502025-04-17T00:30:18.480ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4512025-04-17T00:30:18.482ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4522025-04-17T00:30:18.482ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4532025-04-17T00:30:18.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4542025-04-17T00:30:18.491ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4552025-04-17T00:30:18.491ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4562025-04-17T00:30:18.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4572025-04-17T00:30:18.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4582025-04-17T00:30:18.512ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
4592025-04-17T00:30:18.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4602025-04-17T00:30:18.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4612025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4622025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
4632025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4642025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4652025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4662025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4672025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4682025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4692025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4702025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4712025-04-17T00:30:18.525ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
4722025-04-17T00:30:18.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4732025-04-17T00:30:18.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4742025-04-17T00:30:18.528ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
4752025-04-17T00:30:18.528ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4762025-04-17T00:30:18.542ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 288bc735-c7d8-4f21-bc34-514e05fb41fa roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
4772025-04-17T00:30:18.542ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 288bc735-c7d8-4f21-bc34-514e05fb41fa resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
4782025-04-17T00:30:18.542ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1b07027f-a8bf-4cea-ae5c-e427cbdc5e80", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
4792025-04-17T00:30:18.542ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4802025-04-17T00:30:18.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4812025-04-17T00:30:18.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4822025-04-17T00:30:18.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4832025-04-17T00:30:18.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 5342 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 288bc735-c7d8-4f21-bc34-514e05fb41fa response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
4842025-04-17T00:30:18.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4852025-04-17T00:30:18.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4862025-04-17T00:30:18.554ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4872025-04-17T00:30:18.554ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4882025-04-17T00:30:18.555ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "51c4d10e-f36d-4853-a1a9-67eb221d9647", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
4892025-04-17T00:30:18.557ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 595.94806ms iteration = 19 status = {}
4902025-04-17T00:30:18.583ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4912025-04-17T00:30:18.583ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "64d0684b-2ff3-496b-a8d0-203966d8b185", "content-length": "691", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
4922025-04-17T00:30:18.583ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
4932025-04-17T00:30:18.583ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4942025-04-17T00:30:18.583ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4952025-04-17T00:30:18.584ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 623.139915ms iteration = 19 status = {}
4962025-04-17T00:30:18.637ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "548676d0-3457-46c9-8620-def2ad7ec37f", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
4972025-04-17T00:30:18.637ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
4982025-04-17T00:30:18.637ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4992025-04-17T00:30:18.638ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd09cc92-3320-4e2e-8e3a-6e3b1c129f1a", "content-length": "682", "date": "Thu, 17 Apr 2025 00:30:18 GMT"} })
5002025-04-17T00:30:18.638ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5012025-04-17T00:30:18.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5022025-04-17T00:30:18.641ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5032025-04-17T00:30:18.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5042025-04-17T00:30:18.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5052025-04-17T00:30:18.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5062025-04-17T00:30:18.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5072025-04-17T00:30:18.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5082025-04-17T00:30:18.655ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 693.6353ms iteration = 19 status = {}
5092025-04-17T00:30:19.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6c3fde19-48eb-4826-9b3e-a79c3f5e7c4d roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5102025-04-17T00:30:19.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6c3fde19-48eb-4826-9b3e-a79c3f5e7c4d resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5112025-04-17T00:30:19.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2546 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6c3fde19-48eb-4826-9b3e-a79c3f5e7c4d response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5122025-04-17T00:30:20.590ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ae7d4b62-9352-4474-8b08-d877847cbae9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5132025-04-17T00:30:20.591ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ae7d4b62-9352-4474-8b08-d877847cbae9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5142025-04-17T00:30:20.592ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1991 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ae7d4b62-9352-4474-8b08-d877847cbae9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5152025-04-17T00:30:21.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 844e24e2-7d78-4b64-be58-320832f69e5a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5162025-04-17T00:30:21.613ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 844e24e2-7d78-4b64-be58-320832f69e5a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5172025-04-17T00:30:21.616ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2630 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 844e24e2-7d78-4b64-be58-320832f69e5a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5182025-04-17T00:30:21.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 462 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 29f4b3d0-caf1-47dd-a9ba-bfbf061e73d0 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
5192025-04-17T00:30:22.635ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6e8ec457-e6a5-4f38-80f4-5d5237d96d85 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5202025-04-17T00:30:22.636ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6e8ec457-e6a5-4f38-80f4-5d5237d96d85 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5212025-04-17T00:30:22.638ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2550 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6e8ec457-e6a5-4f38-80f4-5d5237d96d85 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5222025-04-17T00:30:23.656ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1e657042-b0fd-4587-870a-44288d5d4880 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5232025-04-17T00:30:23.657ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1e657042-b0fd-4587-870a-44288d5d4880 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5242025-04-17T00:30:23.659ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2601 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1e657042-b0fd-4587-870a-44288d5d4880 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5252025-04-17T00:30:24.678ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e2b5f64c-7748-4362-a624-76d063b308fd roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5262025-04-17T00:30:24.678ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e2b5f64c-7748-4362-a624-76d063b308fd resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5272025-04-17T00:30:24.680ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2328 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e2b5f64c-7748-4362-a624-76d063b308fd response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5282025-04-17T00:30:25.698ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ac7c5f2a-235c-4e2b-9b9b-3036ec117b1b roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5292025-04-17T00:30:25.698ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ac7c5f2a-235c-4e2b-9b9b-3036ec117b1b resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5302025-04-17T00:30:25.700ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2346 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ac7c5f2a-235c-4e2b-9b9b-3036ec117b1b response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5312025-04-17T00:30:26.718ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 608d053c-bb64-4060-843c-e709cbb4a39c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5322025-04-17T00:30:26.718ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 608d053c-bb64-4060-843c-e709cbb4a39c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5332025-04-17T00:30:26.720ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 608d053c-bb64-4060-843c-e709cbb4a39c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5342025-04-17T00:30:27.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 83cf0ddb-cfff-4b64-b763-8e288e05f076 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5352025-04-17T00:30:27.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 83cf0ddb-cfff-4b64-b763-8e288e05f076 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5362025-04-17T00:30:27.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1918 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 83cf0ddb-cfff-4b64-b763-8e288e05f076 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5372025-04-17T00:30:27.960ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
5382025-04-17T00:30:27.960ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
5392025-04-17T00:30:27.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5402025-04-17T00:30:27.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5412025-04-17T00:30:27.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5422025-04-17T00:30:27.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5432025-04-17T00:30:27.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5442025-04-17T00:30:27.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5452025-04-17T00:30:27.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5462025-04-17T00:30:27.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5472025-04-17T00:30:27.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5482025-04-17T00:30:27.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5492025-04-17T00:30:27.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5502025-04-17T00:30:27.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
5512025-04-17T00:30:27.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5522025-04-17T00:30:27.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5532025-04-17T00:30:27.982ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
5542025-04-17T00:30:27.982ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.172539ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
5552025-04-17T00:30:27.990ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5562025-04-17T00:30:27.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5572025-04-17T00:30:28.026ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.779097ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5582025-04-17T00:30:28.761ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4cbf4e2c-c6a2-4c83-b946-6ba3276c1476 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5592025-04-17T00:30:28.762ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4cbf4e2c-c6a2-4c83-b946-6ba3276c1476 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5602025-04-17T00:30:28.764ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2538 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4cbf4e2c-c6a2-4c83-b946-6ba3276c1476 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5612025-04-17T00:30:29.783ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5c34395b-4cfd-4a46-938c-90381928d193 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5622025-04-17T00:30:29.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5c34395b-4cfd-4a46-938c-90381928d193 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5632025-04-17T00:30:29.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2672 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5c34395b-4cfd-4a46-938c-90381928d193 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5642025-04-17T00:30:30.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 68834ca7-32ea-494f-a737-61468269af46 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5652025-04-17T00:30:30.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 68834ca7-32ea-494f-a737-61468269af46 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5662025-04-17T00:30:30.808ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 68834ca7-32ea-494f-a737-61468269af46 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5672025-04-17T00:30:31.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = a45e8c11-b074-43f8-a58c-12ff1679d5b9 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
5682025-04-17T00:30:31.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = a45e8c11-b074-43f8-a58c-12ff1679d5b9 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
5692025-04-17T00:30:31.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3653 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = a45e8c11-b074-43f8-a58c-12ff1679d5b9 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
5702025-04-17T00:30:31.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 4e3a6e1e-932a-4512-b71e-dc24087b20b8 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
5712025-04-17T00:30:31.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 4e3a6e1e-932a-4512-b71e-dc24087b20b8 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
5722025-04-17T00:30:31.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2143 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 4e3a6e1e-932a-4512-b71e-dc24087b20b8 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
5732025-04-17T00:30:31.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 469 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 3decc9df-12ac-4c11-8a59-c8fda426fb23 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
5742025-04-17T00:30:31.826ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = dd76554a-c7ee-42df-8a6f-dbd6271062b1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5752025-04-17T00:30:31.827ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = dd76554a-c7ee-42df-8a6f-dbd6271062b1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5762025-04-17T00:30:31.828ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2105 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = dd76554a-c7ee-42df-8a6f-dbd6271062b1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5772025-04-17T00:30:32.848ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 41e560af-9437-40bb-bbd5-a2788e25b19c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5782025-04-17T00:30:32.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 41e560af-9437-40bb-bbd5-a2788e25b19c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5792025-04-17T00:30:32.850ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2186 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 41e560af-9437-40bb-bbd5-a2788e25b19c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5802025-04-17T00:30:33.869ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a6a869f0-871c-4d85-a9c8-a04c62189ce5 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5812025-04-17T00:30:33.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a6a869f0-871c-4d85-a9c8-a04c62189ce5 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5822025-04-17T00:30:33.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2482 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a6a869f0-871c-4d85-a9c8-a04c62189ce5 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5832025-04-17T00:30:34.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b4e8df63-4729-4b07-a5a7-7d2482deb584 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5842025-04-17T00:30:34.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b4e8df63-4729-4b07-a5a7-7d2482deb584 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5852025-04-17T00:30:34.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1860 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b4e8df63-4729-4b07-a5a7-7d2482deb584 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5862025-04-17T00:30:35.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7a5c06b6-fe5e-4238-b4bd-a68d8c73b350 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5872025-04-17T00:30:35.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7a5c06b6-fe5e-4238-b4bd-a68d8c73b350 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5882025-04-17T00:30:35.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2028 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7a5c06b6-fe5e-4238-b4bd-a68d8c73b350 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5892025-04-17T00:30:36.936ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e7d3e913-60ef-466c-aa30-942c9eb15908 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5902025-04-17T00:30:36.937ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e7d3e913-60ef-466c-aa30-942c9eb15908 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5912025-04-17T00:30:36.939ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2401 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e7d3e913-60ef-466c-aa30-942c9eb15908 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5922025-04-17T00:30:37.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eda0e20e-c0e4-4a18-aaf8-0c90cb594ccc roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5932025-04-17T00:30:37.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eda0e20e-c0e4-4a18-aaf8-0c90cb594ccc resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5942025-04-17T00:30:37.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2090 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eda0e20e-c0e4-4a18-aaf8-0c90cb594ccc response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
5952025-04-17T00:30:37.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
5962025-04-17T00:30:37.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
5972025-04-17T00:30:37.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5982025-04-17T00:30:37.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5992025-04-17T00:30:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6002025-04-17T00:30:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6012025-04-17T00:30:37.993ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6022025-04-17T00:30:37.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6032025-04-17T00:30:37.995ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.112499ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6042025-04-17T00:30:38.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6052025-04-17T00:30:38.003ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6062025-04-17T00:30:38.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6072025-04-17T00:30:38.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6082025-04-17T00:30:38.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6092025-04-17T00:30:38.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6102025-04-17T00:30:38.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6112025-04-17T00:30:38.009ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
6122025-04-17T00:30:38.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6132025-04-17T00:30:38.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6142025-04-17T00:30:38.018ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
6152025-04-17T00:30:38.018ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.396986ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
6162025-04-17T00:30:38.981ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = adc341a6-09d8-4842-9321-089028fffc8f roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6172025-04-17T00:30:38.981ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = adc341a6-09d8-4842-9321-089028fffc8f resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6182025-04-17T00:30:38.983ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1993 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = adc341a6-09d8-4842-9321-089028fffc8f response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6192025-04-17T00:30:40.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 66ebe040-88f5-4a1d-92e4-e1eff17388d6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6202025-04-17T00:30:40.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 66ebe040-88f5-4a1d-92e4-e1eff17388d6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6212025-04-17T00:30:40.003ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1887 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 66ebe040-88f5-4a1d-92e4-e1eff17388d6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6222025-04-17T00:30:41.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1c3e19dd-9198-4e5b-b97d-ef1320952fe4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6232025-04-17T00:30:41.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1c3e19dd-9198-4e5b-b97d-ef1320952fe4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6242025-04-17T00:30:41.027ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2048 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1c3e19dd-9198-4e5b-b97d-ef1320952fe4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6252025-04-17T00:30:41.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 462 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 2933a177-8728-452e-9b79-38419ead211b response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
6262025-04-17T00:30:42.047ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 51830ad5-91a8-46eb-b2e2-78d1ab0a935e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6272025-04-17T00:30:42.047ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 51830ad5-91a8-46eb-b2e2-78d1ab0a935e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6282025-04-17T00:30:42.049ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2064 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 51830ad5-91a8-46eb-b2e2-78d1ab0a935e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6292025-04-17T00:30:43.070ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0b2e472a-3d1f-405d-8fb2-75449865cd78 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6302025-04-17T00:30:43.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0b2e472a-3d1f-405d-8fb2-75449865cd78 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6312025-04-17T00:30:43.072ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2191 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0b2e472a-3d1f-405d-8fb2-75449865cd78 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6322025-04-17T00:30:44.092ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d97bbc90-5629-4a5c-8af8-d91daf64508c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6332025-04-17T00:30:44.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d97bbc90-5629-4a5c-8af8-d91daf64508c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6342025-04-17T00:30:44.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1872 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d97bbc90-5629-4a5c-8af8-d91daf64508c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6352025-04-17T00:30:45.114ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9176c312-e925-43e3-a910-98365ef12cbe roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6362025-04-17T00:30:45.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9176c312-e925-43e3-a910-98365ef12cbe resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6372025-04-17T00:30:45.117ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2850 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9176c312-e925-43e3-a910-98365ef12cbe response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6382025-04-17T00:30:46.137ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b2dd456f-1a0a-4098-98bd-173fc2de3a1e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6392025-04-17T00:30:46.137ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b2dd456f-1a0a-4098-98bd-173fc2de3a1e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6402025-04-17T00:30:46.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1933 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b2dd456f-1a0a-4098-98bd-173fc2de3a1e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6412025-04-17T00:30:47.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 064a269a-ff77-48bd-887d-7107419668b2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6422025-04-17T00:30:47.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 064a269a-ff77-48bd-887d-7107419668b2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6432025-04-17T00:30:47.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 064a269a-ff77-48bd-887d-7107419668b2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
6442025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
6452025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
6462025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6472025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6482025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
6492025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6502025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
6512025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
6522025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6532025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6542025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
6552025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6562025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
6572025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6582025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
6592025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
6602025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6612025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
6622025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
6632025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6642025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
6652025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6662025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
6672025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
6682025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6692025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6702025-04-17T00:30:47.960ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6712025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
6722025-04-17T00:30:47.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6732025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6742025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
6752025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6762025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
6772025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6782025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
6792025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6802025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
6812025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6822025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
6832025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6842025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6852025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6862025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6872025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6882025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6892025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6902025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6912025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6922025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
6932025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6942025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6952025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6962025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6972025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6982025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6992025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7002025-04-17T00:30:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7012025-04-17T00:30:47.963ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1773, tv_nsec: 634588270 } })
7022025-04-17T00:30:47.963ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
7032025-04-17T00:30:47.964ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1773, tv_nsec: 635540834 } })
7042025-04-17T00:30:47.964ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.466407ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7052025-04-17T00:30:47.964ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
7062025-04-17T00:30:47.985ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7072025-04-17T00:30:47.985ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7082025-04-17T00:30:47.985ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7092025-04-17T00:30:47.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7102025-04-17T00:30:48.014ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 53.178889ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
7112025-04-17T00:30:48.015ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7122025-04-17T00:30:48.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7132025-04-17T00:30:48.028ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142025-04-17T00:30:48.028ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.564622ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
7152025-04-17T00:30:48.030ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7162025-04-17T00:30:48.030ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7172025-04-17T00:30:48.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7182025-04-17T00:30:48.033ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7192025-04-17T00:30:48.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7202025-04-17T00:30:48.034ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7212025-04-17T00:30:48.034ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7222025-04-17T00:30:48.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7232025-04-17T00:30:48.036ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7242025-04-17T00:30:48.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7252025-04-17T00:30:48.038ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7262025-04-17T00:30:48.038ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 77.567864ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7272025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7282025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7292025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7302025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7312025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7322025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7332025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7342025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7352025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7362025-04-17T00:30:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7372025-04-17T00:30:48.040ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 79.521532ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
7382025-04-17T00:30:48.040ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7392025-04-17T00:30:48.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7402025-04-17T00:30:48.041ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7412025-04-17T00:30:48.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7422025-04-17T00:30:48.041ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7432025-04-17T00:30:48.042ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7442025-04-17T00:30:48.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7452025-04-17T00:30:48.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7462025-04-17T00:30:48.042ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7472025-04-17T00:30:48.043ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1773, tv_nsec: 714557155 } })
7482025-04-17T00:30:48.043ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
7492025-04-17T00:30:48.044ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
7502025-04-17T00:30:48.044ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7512025-04-17T00:30:48.044ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da64bdf8-cf2d-4f12-993b-79a6cf1d606e", "content-length": "26", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
7522025-04-17T00:30:48.044ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7532025-04-17T00:30:48.044ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7542025-04-17T00:30:48.044ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7552025-04-17T00:30:48.044ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7562025-04-17T00:30:48.045ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7572025-04-17T00:30:48.045ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7582025-04-17T00:30:48.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7592025-04-17T00:30:48.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7602025-04-17T00:30:48.046ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7612025-04-17T00:30:48.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7622025-04-17T00:30:48.046ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7632025-04-17T00:30:48.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7642025-04-17T00:30:48.049ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 88.135309ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7652025-04-17T00:30:48.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7662025-04-17T00:30:48.055ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
7672025-04-17T00:30:48.055ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7682025-04-17T00:30:48.055ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e5b7521-5ffb-46da-8740-e5cf81905709", "content-length": "26", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
7692025-04-17T00:30:48.055ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7702025-04-17T00:30:48.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7712025-04-17T00:30:48.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7722025-04-17T00:30:48.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7732025-04-17T00:30:48.058ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7742025-04-17T00:30:48.059ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7752025-04-17T00:30:48.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7762025-04-17T00:30:48.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7772025-04-17T00:30:48.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7782025-04-17T00:30:48.077ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7792025-04-17T00:30:48.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7802025-04-17T00:30:48.084ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 123.822109ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7812025-04-17T00:30:48.089ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 129.180244ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7822025-04-17T00:30:48.093ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7832025-04-17T00:30:48.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7842025-04-17T00:30:48.096ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7852025-04-17T00:30:48.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7862025-04-17T00:30:48.099ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7872025-04-17T00:30:48.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7882025-04-17T00:30:48.102ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7892025-04-17T00:30:48.102ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7902025-04-17T00:30:48.102ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 141.105054ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7912025-04-17T00:30:48.104ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7922025-04-17T00:30:48.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7932025-04-17T00:30:48.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7942025-04-17T00:30:48.109ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7952025-04-17T00:30:48.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7962025-04-17T00:30:48.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
7972025-04-17T00:30:48.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7982025-04-17T00:30:48.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7992025-04-17T00:30:48.115ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 154.990364ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
8002025-04-17T00:30:48.130ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
8012025-04-17T00:30:48.130ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 169.484869ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
8022025-04-17T00:30:48.135ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8032025-04-17T00:30:48.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8042025-04-17T00:30:48.141ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8052025-04-17T00:30:48.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
8062025-04-17T00:30:48.143ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8072025-04-17T00:30:48.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8082025-04-17T00:30:48.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8092025-04-17T00:30:48.147ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8102025-04-17T00:30:48.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8112025-04-17T00:30:48.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
8122025-04-17T00:30:48.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8132025-04-17T00:30:48.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8142025-04-17T00:30:48.150ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 189.704688ms iteration = 20 status = null
8152025-04-17T00:30:48.151ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 190.221911ms iteration = 20 status = 0
8162025-04-17T00:30:48.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90a71e78-6d0f-4644-8ae2-dd7e8b2dfda1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8172025-04-17T00:30:48.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90a71e78-6d0f-4644-8ae2-dd7e8b2dfda1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8182025-04-17T00:30:48.192ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
8192025-04-17T00:30:48.192ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 230.979738ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
8202025-04-17T00:30:48.218ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 258.016999ms iteration = 20 status = {}
8212025-04-17T00:30:48.227ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
8222025-04-17T00:30:48.228ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8232025-04-17T00:30:48.228ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2824187-1817-4359-b911-c5629a76d53c", "content-length": "26", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8242025-04-17T00:30:48.228ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8252025-04-17T00:30:48.230ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
8262025-04-17T00:30:48.237ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 276.490614ms iteration = 22 status = {"physical_disks_added":0}
8272025-04-17T00:30:48.238ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8282025-04-17T00:30:48.238ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8292025-04-17T00:30:48.238ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3c72516-f35b-40d2-b621-894db6aabb1b", "content-length": "372", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8302025-04-17T00:30:48.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8312025-04-17T00:30:48.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8322025-04-17T00:30:48.247ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 65368 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90a71e78-6d0f-4644-8ae2-dd7e8b2dfda1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8332025-04-17T00:30:48.265ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8342025-04-17T00:30:48.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8352025-04-17T00:30:48.294ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
8362025-04-17T00:30:48.294ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8372025-04-17T00:30:48.298ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8382025-04-17T00:30:48.300ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8392025-04-17T00:30:48.300ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8402025-04-17T00:30:48.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8412025-04-17T00:30:48.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422025-04-17T00:30:48.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8432025-04-17T00:30:48.334ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8442025-04-17T00:30:48.337ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
8452025-04-17T00:30:48.337ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8462025-04-17T00:30:48.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8472025-04-17T00:30:48.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8482025-04-17T00:30:48.337ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8492025-04-17T00:30:48.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8502025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8512025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
8522025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8532025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8542025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8552025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8562025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8572025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8582025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8592025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8602025-04-17T00:30:48.349ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
8612025-04-17T00:30:48.349ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8622025-04-17T00:30:48.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8632025-04-17T00:30:48.352ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
8642025-04-17T00:30:48.352ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8652025-04-17T00:30:48.392ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5d3fe73b-e10c-4b1e-a3ab-7f1db1f41f06", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8662025-04-17T00:30:48.392ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8672025-04-17T00:30:48.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8682025-04-17T00:30:48.395ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8692025-04-17T00:30:48.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8702025-04-17T00:30:48.398ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8712025-04-17T00:30:48.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8722025-04-17T00:30:48.408ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
8732025-04-17T00:30:48.408ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8742025-04-17T00:30:48.409ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "049aec45-0f83-4693-9de4-af60288be0be", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8752025-04-17T00:30:48.410ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 449.564513ms iteration = 20 status = {}
8762025-04-17T00:30:48.444ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8772025-04-17T00:30:48.448ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a85c750c-46c0-4d1f-98ab-f46e1d00e5b5", "content-length": "691", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8782025-04-17T00:30:48.448ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
8792025-04-17T00:30:48.448ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
8802025-04-17T00:30:48.448ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
8812025-04-17T00:30:48.450ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 489.420678ms iteration = 20 status = {}
8822025-04-17T00:30:48.515ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "365524e2-f660-4e61-bcb8-aff2f95ac2a1", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8832025-04-17T00:30:48.515ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
8842025-04-17T00:30:48.515ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8852025-04-17T00:30:48.519ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "994ef9e6-8737-4ba2-a74f-27290b48efba", "content-length": "682", "date": "Thu, 17 Apr 2025 00:30:48 GMT"} })
8862025-04-17T00:30:48.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8872025-04-17T00:30:48.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8882025-04-17T00:30:48.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8892025-04-17T00:30:48.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8902025-04-17T00:30:48.528ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8912025-04-17T00:30:48.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8922025-04-17T00:30:48.531ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8932025-04-17T00:30:48.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8942025-04-17T00:30:48.538ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 576.974342ms iteration = 20 status = {}
8952025-04-17T00:30:49.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca35622f-bddf-4ff9-97ab-f29d5c7e4380 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8962025-04-17T00:30:49.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca35622f-bddf-4ff9-97ab-f29d5c7e4380 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8972025-04-17T00:30:49.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2786 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca35622f-bddf-4ff9-97ab-f29d5c7e4380 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8982025-04-17T00:30:50.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3b87cf7b-f93c-4c7c-b45d-00586984845b roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
8992025-04-17T00:30:50.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3b87cf7b-f93c-4c7c-b45d-00586984845b resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9002025-04-17T00:30:50.294ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1872 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3b87cf7b-f93c-4c7c-b45d-00586984845b response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9012025-04-17T00:30:51.316ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 57c74930-f3bf-4786-a169-7c9a0e1b7e77 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9022025-04-17T00:30:51.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 57c74930-f3bf-4786-a169-7c9a0e1b7e77 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9032025-04-17T00:30:51.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2564 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 57c74930-f3bf-4786-a169-7c9a0e1b7e77 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9042025-04-17T00:30:51.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 501 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = c8fb1e71-089c-42ec-9ebb-5d0584da5a3a response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
9052025-04-17T00:30:52.339ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 30b4c4ed-e81e-4150-9c83-7935cfadeabb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9062025-04-17T00:30:52.339ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 30b4c4ed-e81e-4150-9c83-7935cfadeabb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9072025-04-17T00:30:52.341ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2780 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 30b4c4ed-e81e-4150-9c83-7935cfadeabb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9082025-04-17T00:30:53.362ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ed2d4008-e09b-4845-83df-e9955d4c08b3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9092025-04-17T00:30:53.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ed2d4008-e09b-4845-83df-e9955d4c08b3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9102025-04-17T00:30:53.365ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2915 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ed2d4008-e09b-4845-83df-e9955d4c08b3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9112025-04-17T00:30:54.385ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8bc5b7ae-7b27-4f73-a458-43859bceec61 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9122025-04-17T00:30:54.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8bc5b7ae-7b27-4f73-a458-43859bceec61 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9132025-04-17T00:30:54.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2183 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8bc5b7ae-7b27-4f73-a458-43859bceec61 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9142025-04-17T00:30:55.408ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5b83da47-dc5f-4550-abee-7fc4338c60c5 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9152025-04-17T00:30:55.408ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5b83da47-dc5f-4550-abee-7fc4338c60c5 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9162025-04-17T00:30:55.409ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1646 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5b83da47-dc5f-4550-abee-7fc4338c60c5 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9172025-04-17T00:30:56.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2de99279-ac87-4b9f-8028-fb0ecbbaddb1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9182025-04-17T00:30:56.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2de99279-ac87-4b9f-8028-fb0ecbbaddb1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9192025-04-17T00:30:56.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2189 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2de99279-ac87-4b9f-8028-fb0ecbbaddb1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9202025-04-17T00:30:57.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8c73ad25-6b53-4388-bee3-983f4bec5cca roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9212025-04-17T00:30:57.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8c73ad25-6b53-4388-bee3-983f4bec5cca resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9222025-04-17T00:30:57.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2298 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8c73ad25-6b53-4388-bee3-983f4bec5cca response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9232025-04-17T00:30:57.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
9242025-04-17T00:30:57.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
9252025-04-17T00:30:57.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9262025-04-17T00:30:57.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9272025-04-17T00:30:57.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9282025-04-17T00:30:57.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9292025-04-17T00:30:57.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9302025-04-17T00:30:57.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9312025-04-17T00:30:57.998ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.281309ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9322025-04-17T00:30:57.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9332025-04-17T00:30:58.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9342025-04-17T00:30:58.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9352025-04-17T00:30:58.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9362025-04-17T00:30:58.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9372025-04-17T00:30:58.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9382025-04-17T00:30:58.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9392025-04-17T00:30:58.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
9402025-04-17T00:30:58.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9412025-04-17T00:30:58.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9422025-04-17T00:30:58.018ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
9432025-04-17T00:30:58.018ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.853912ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
9442025-04-17T00:30:58.473ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 42b6048f-ddb5-4dba-af03-048c0829fe93 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9452025-04-17T00:30:58.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 42b6048f-ddb5-4dba-af03-048c0829fe93 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9462025-04-17T00:30:58.474ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1853 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 42b6048f-ddb5-4dba-af03-048c0829fe93 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9472025-04-17T00:30:59.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7815ea70-7959-4ff9-9b64-71c0306512b3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9482025-04-17T00:30:59.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7815ea70-7959-4ff9-9b64-71c0306512b3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9492025-04-17T00:30:59.495ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1450 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7815ea70-7959-4ff9-9b64-71c0306512b3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9502025-04-17T00:31:00.515ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90c6fe76-0355-4e69-8916-457ae6aee5c6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9512025-04-17T00:31:00.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90c6fe76-0355-4e69-8916-457ae6aee5c6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9522025-04-17T00:31:00.517ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2359 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90c6fe76-0355-4e69-8916-457ae6aee5c6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9532025-04-17T00:31:01.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6d275bc0-12bc-49f5-9b40-29072bc969fb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9542025-04-17T00:31:01.537ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6d275bc0-12bc-49f5-9b40-29072bc969fb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9552025-04-17T00:31:01.539ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2268 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6d275bc0-12bc-49f5-9b40-29072bc969fb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9562025-04-17T00:31:01.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 335e4e0a-b67e-452f-9c35-f3b93b25fdd7 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
9572025-04-17T00:31:01.638ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 335e4e0a-b67e-452f-9c35-f3b93b25fdd7 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
9582025-04-17T00:31:01.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2714 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 335e4e0a-b67e-452f-9c35-f3b93b25fdd7 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
9592025-04-17T00:31:01.641ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 0394fbdc-0179-45a2-96c8-db2c4b1e73c3 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
9602025-04-17T00:31:01.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 0394fbdc-0179-45a2-96c8-db2c4b1e73c3 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
9612025-04-17T00:31:01.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:39449 req_id = 0394fbdc-0179-45a2-96c8-db2c4b1e73c3 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
9622025-04-17T00:31:01.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 760 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = bfd0e245-be77-487d-b753-acbf7e369229 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
9632025-04-17T00:31:02.558ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4c46b97b-ab80-4717-96bd-38c82f26ed77 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9642025-04-17T00:31:02.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4c46b97b-ab80-4717-96bd-38c82f26ed77 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9652025-04-17T00:31:02.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1780 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4c46b97b-ab80-4717-96bd-38c82f26ed77 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9662025-04-17T00:31:03.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 258e2706-4eb7-46e2-a47d-2e56cfa81151 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9672025-04-17T00:31:03.582ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 258e2706-4eb7-46e2-a47d-2e56cfa81151 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9682025-04-17T00:31:03.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2122 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 258e2706-4eb7-46e2-a47d-2e56cfa81151 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9692025-04-17T00:31:04.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 12eba9eb-0bed-4423-a03a-0ab8a678ead4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9702025-04-17T00:31:04.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 12eba9eb-0bed-4423-a03a-0ab8a678ead4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9712025-04-17T00:31:04.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2507 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 12eba9eb-0bed-4423-a03a-0ab8a678ead4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9722025-04-17T00:31:05.628ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a77e0387-eda9-4203-8e8c-78b9821c1160 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9732025-04-17T00:31:05.628ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a77e0387-eda9-4203-8e8c-78b9821c1160 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9742025-04-17T00:31:05.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2745 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a77e0387-eda9-4203-8e8c-78b9821c1160 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9752025-04-17T00:31:06.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d1e23c6d-117e-429a-8126-a75350244cc5 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9762025-04-17T00:31:06.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d1e23c6d-117e-429a-8126-a75350244cc5 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9772025-04-17T00:31:06.653ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2367 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d1e23c6d-117e-429a-8126-a75350244cc5 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9782025-04-17T00:31:07.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0e610c3b-9d8c-4f23-b152-cf44a9bd5c2a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9792025-04-17T00:31:07.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0e610c3b-9d8c-4f23-b152-cf44a9bd5c2a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9802025-04-17T00:31:07.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1667 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0e610c3b-9d8c-4f23-b152-cf44a9bd5c2a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
9812025-04-17T00:31:07.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
9822025-04-17T00:31:07.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
9832025-04-17T00:31:07.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9842025-04-17T00:31:07.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9852025-04-17T00:31:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9862025-04-17T00:31:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9872025-04-17T00:31:07.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9882025-04-17T00:31:07.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9892025-04-17T00:31:07.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9902025-04-17T00:31:07.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9912025-04-17T00:31:07.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9922025-04-17T00:31:07.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9932025-04-17T00:31:07.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9942025-04-17T00:31:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
9952025-04-17T00:31:07.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9962025-04-17T00:31:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9972025-04-17T00:31:07.980ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
9982025-04-17T00:31:07.980ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.779788ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
9992025-04-17T00:31:07.991ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10002025-04-17T00:31:07.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10012025-04-17T00:31:08.021ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.653538ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10022025-04-17T00:31:08.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 936870b7-9989-487f-b3a6-35fb01bf7e28 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10032025-04-17T00:31:08.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 936870b7-9989-487f-b3a6-35fb01bf7e28 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10042025-04-17T00:31:08.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2331 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 936870b7-9989-487f-b3a6-35fb01bf7e28 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10052025-04-17T00:31:09.717ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f4ea0c60-ddae-4657-bbc6-7bd2839f8ea9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10062025-04-17T00:31:09.717ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f4ea0c60-ddae-4657-bbc6-7bd2839f8ea9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10072025-04-17T00:31:09.719ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2239 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f4ea0c60-ddae-4657-bbc6-7bd2839f8ea9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10082025-04-17T00:31:10.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e3a545a0-4b06-40af-a621-d3c4c9e65828 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10092025-04-17T00:31:10.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e3a545a0-4b06-40af-a621-d3c4c9e65828 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10102025-04-17T00:31:10.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1527 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e3a545a0-4b06-40af-a621-d3c4c9e65828 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10112025-04-17T00:31:11.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 523 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = db9e291b-017d-446e-a65d-cb5b5ba4882a response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
10122025-04-17T00:31:11.761ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3563d302-5e34-458b-8cf4-fe4d02043443 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10132025-04-17T00:31:11.762ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3563d302-5e34-458b-8cf4-fe4d02043443 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10142025-04-17T00:31:11.764ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2539 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3563d302-5e34-458b-8cf4-fe4d02043443 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10152025-04-17T00:31:12.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b7f3d1dd-347d-4814-b302-2d89b0908ba6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10162025-04-17T00:31:12.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b7f3d1dd-347d-4814-b302-2d89b0908ba6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10172025-04-17T00:31:12.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2532 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b7f3d1dd-347d-4814-b302-2d89b0908ba6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10182025-04-17T00:31:13.806ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 10bc7e3c-382d-426e-ac80-c33ff237cada roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10192025-04-17T00:31:13.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 10bc7e3c-382d-426e-ac80-c33ff237cada resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10202025-04-17T00:31:13.808ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1972 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 10bc7e3c-382d-426e-ac80-c33ff237cada response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10212025-04-17T00:31:14.828ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f2a563b3-57e5-430e-abe1-81b53f6ac98c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10222025-04-17T00:31:14.828ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f2a563b3-57e5-430e-abe1-81b53f6ac98c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10232025-04-17T00:31:14.830ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1997 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f2a563b3-57e5-430e-abe1-81b53f6ac98c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10242025-04-17T00:31:15.850ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3caf7ed6-bb1d-48a8-aa7d-8b98529e9e20 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10252025-04-17T00:31:15.850ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3caf7ed6-bb1d-48a8-aa7d-8b98529e9e20 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10262025-04-17T00:31:15.852ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2218 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3caf7ed6-bb1d-48a8-aa7d-8b98529e9e20 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10272025-04-17T00:31:16.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2d6614b2-1b57-42c8-a360-5f000084f47a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10282025-04-17T00:31:16.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2d6614b2-1b57-42c8-a360-5f000084f47a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10292025-04-17T00:31:16.875ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2312 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2d6614b2-1b57-42c8-a360-5f000084f47a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10302025-04-17T00:31:17.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 97081e61-1969-406e-bd6c-20cd87717107 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10312025-04-17T00:31:17.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 97081e61-1969-406e-bd6c-20cd87717107 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10322025-04-17T00:31:17.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1892 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 97081e61-1969-406e-bd6c-20cd87717107 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
10332025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
10342025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10352025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
10362025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10372025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
10382025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10392025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
10402025-04-17T00:31:17.961ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
10412025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10422025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
10432025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
10442025-04-17T00:31:17.961ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
10452025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10462025-04-17T00:31:17.961ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
10472025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
10482025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10492025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
10502025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10512025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
10522025-04-17T00:31:17.961ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3
10532025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.98\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3"}
10542025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
10552025-04-17T00:31:17.961ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-17 00:21:17.961502966 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
10562025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
10572025-04-17T00:31:17.961ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
10582025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
10592025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10602025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
10612025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
10622025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
10632025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10642025-04-17T00:31:17.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10652025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
10662025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10672025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
10682025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
10692025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10702025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
10712025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10722025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
10732025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10742025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
10752025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10762025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
10772025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10782025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Timeout
10792025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
10802025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10812025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
10822025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10832025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
10842025-04-17T00:31:17.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10852025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10862025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
10872025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
10882025-04-17T00:31:17.962ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
10892025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10902025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10912025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
10922025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
10932025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
10942025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10952025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
10962025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10972025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
10982025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10992025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11002025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
11012025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
11022025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11032025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
11042025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
11052025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11062025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
11072025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11082025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
11092025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
11102025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
11112025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
11122025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11132025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
11142025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11152025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
11162025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11172025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11182025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
11192025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
11202025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11212025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
11222025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11232025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11242025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11252025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11262025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11272025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
11282025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11292025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11302025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11312025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11322025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11332025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11342025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11352025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11362025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11372025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11382025-04-17T00:31:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11392025-04-17T00:31:17.962ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b3e44f3-f539-46b4-bb67-d442c7ebd3cf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4488d95c-7f86-4ca6-ab72-2e345a6a57bb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1803, tv_nsec: 633498332 } })
11402025-04-17T00:31:17.962ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99145431-02b3-4b6e-aa5d-968f96332449.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1803, tv_nsec: 633557334 } })
11412025-04-17T00:31:17.963ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
11422025-04-17T00:31:17.963ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.327933ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
11432025-04-17T00:31:17.964ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
11442025-04-17T00:31:17.964ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.675257ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
11452025-04-17T00:31:17.966ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11462025-04-17T00:31:17.966ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1803, tv_nsec: 637185365 } })
11472025-04-17T00:31:17.966ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1803, tv_nsec: 637348469 } })
11482025-04-17T00:31:17.966ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
11492025-04-17T00:31:17.967ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
11502025-04-17T00:31:17.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
11512025-04-17T00:31:17.969ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
11522025-04-17T00:31:17.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
11532025-04-17T00:31:17.974ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.402187ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
11542025-04-17T00:31:17.974ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11552025-04-17T00:31:17.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11562025-04-17T00:31:17.983ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11572025-04-17T00:31:17.983ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-17 00:21:17.961502966 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
11582025-04-17T00:31:17.983ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11592025-04-17T00:31:17.983ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 22.213679ms iteration = 11 status = {"expiration":"2025-04-17T00:21:17.961502966Z","pruned":[]}
11602025-04-17T00:31:17.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11612025-04-17T00:31:17.985ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 23.637436ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
11622025-04-17T00:31:17.986ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11632025-04-17T00:31:17.986ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11642025-04-17T00:31:17.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11652025-04-17T00:31:17.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11662025-04-17T00:31:17.989ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11672025-04-17T00:31:17.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11682025-04-17T00:31:17.991ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11692025-04-17T00:31:17.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11702025-04-17T00:31:17.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11712025-04-17T00:31:17.995ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11722025-04-17T00:31:17.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11732025-04-17T00:31:18.029ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
11742025-04-17T00:31:18.029ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 68.215108ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
11752025-04-17T00:31:18.035ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.037265ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
11762025-04-17T00:31:18.035ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772025-04-17T00:31:18.035ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 74.145208ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
11782025-04-17T00:31:18.037ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11792025-04-17T00:31:18.038ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11802025-04-17T00:31:18.038ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11812025-04-17T00:31:18.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11822025-04-17T00:31:18.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11832025-04-17T00:31:18.044ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11842025-04-17T00:31:18.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11852025-04-17T00:31:18.049ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11862025-04-17T00:31:18.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11872025-04-17T00:31:18.050ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11882025-04-17T00:31:18.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11892025-04-17T00:31:18.052ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11902025-04-17T00:31:18.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11912025-04-17T00:31:18.055ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 94.191783ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11922025-04-17T00:31:18.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11932025-04-17T00:31:18.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11942025-04-17T00:31:18.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11952025-04-17T00:31:18.062ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11962025-04-17T00:31:18.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
11972025-04-17T00:31:18.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11982025-04-17T00:31:18.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11992025-04-17T00:31:18.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002025-04-17T00:31:18.066ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12012025-04-17T00:31:18.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12022025-04-17T00:31:18.067ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12032025-04-17T00:31:18.068ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
12042025-04-17T00:31:18.068ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
12052025-04-17T00:31:18.068ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 106.830231ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
12062025-04-17T00:31:18.068ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12072025-04-17T00:31:18.068ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
12082025-04-17T00:31:18.068ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12092025-04-17T00:31:18.068ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12102025-04-17T00:31:18.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12112025-04-17T00:31:18.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12122025-04-17T00:31:18.070ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12132025-04-17T00:31:18.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12142025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12152025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12162025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12172025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
12182025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12192025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12202025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12212025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12222025-04-17T00:31:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12232025-04-17T00:31:18.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12242025-04-17T00:31:18.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12252025-04-17T00:31:18.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12262025-04-17T00:31:18.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
12272025-04-17T00:31:18.074ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12282025-04-17T00:31:18.074ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12292025-04-17T00:31:18.074ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12302025-04-17T00:31:18.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12312025-04-17T00:31:18.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12322025-04-17T00:31:18.074ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 113.273773ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
12332025-04-17T00:31:18.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12342025-04-17T00:31:18.075ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12352025-04-17T00:31:18.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12362025-04-17T00:31:18.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12372025-04-17T00:31:18.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12382025-04-17T00:31:18.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12392025-04-17T00:31:18.080ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12402025-04-17T00:31:18.080ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
12412025-04-17T00:31:18.080ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12422025-04-17T00:31:18.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12432025-04-17T00:31:18.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
12442025-04-17T00:31:18.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12452025-04-17T00:31:18.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12462025-04-17T00:31:18.087ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1803, tv_nsec: 758751587 } })
12472025-04-17T00:31:18.088ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
12482025-04-17T00:31:18.088ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 126.861846ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12492025-04-17T00:31:18.092ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12502025-04-17T00:31:18.092ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12512025-04-17T00:31:18.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12522025-04-17T00:31:18.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12532025-04-17T00:31:18.094ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12542025-04-17T00:31:18.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12552025-04-17T00:31:18.096ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
12562025-04-17T00:31:18.096ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
12572025-04-17T00:31:18.096ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 135.075512ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
12582025-04-17T00:31:18.098ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12592025-04-17T00:31:18.101ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12602025-04-17T00:31:18.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12612025-04-17T00:31:18.101ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12622025-04-17T00:31:18.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12632025-04-17T00:31:18.104ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
12642025-04-17T00:31:18.104ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12652025-04-17T00:31:18.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12662025-04-17T00:31:18.109ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12672025-04-17T00:31:18.111ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12682025-04-17T00:31:18.113ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
12692025-04-17T00:31:18.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
12702025-04-17T00:31:18.114ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
12712025-04-17T00:31:18.114ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 153.148827ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
12722025-04-17T00:31:18.114ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12732025-04-17T00:31:18.114ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12742025-04-17T00:31:18.114ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 152.904941ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12752025-04-17T00:31:18.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12762025-04-17T00:31:18.115ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
12772025-04-17T00:31:18.115ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12782025-04-17T00:31:18.115ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc933e53-d998-4e1a-acac-33f5b512fbdf", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
12792025-04-17T00:31:18.115ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12802025-04-17T00:31:18.117ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12812025-04-17T00:31:18.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12822025-04-17T00:31:18.121ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12832025-04-17T00:31:18.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12842025-04-17T00:31:18.123ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12852025-04-17T00:31:18.123ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12862025-04-17T00:31:18.124ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12872025-04-17T00:31:18.125ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12882025-04-17T00:31:18.125ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12892025-04-17T00:31:18.125ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12902025-04-17T00:31:18.125ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12912025-04-17T00:31:18.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12922025-04-17T00:31:18.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12932025-04-17T00:31:18.130ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12942025-04-17T00:31:18.130ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12952025-04-17T00:31:18.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12962025-04-17T00:31:18.130ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12972025-04-17T00:31:18.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12982025-04-17T00:31:18.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12992025-04-17T00:31:18.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13002025-04-17T00:31:18.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13012025-04-17T00:31:18.133ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13022025-04-17T00:31:18.133ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13032025-04-17T00:31:18.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13042025-04-17T00:31:18.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13052025-04-17T00:31:18.135ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062025-04-17T00:31:18.136ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13072025-04-17T00:31:18.136ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
13082025-04-17T00:31:18.136ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13092025-04-17T00:31:18.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13102025-04-17T00:31:18.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13112025-04-17T00:31:18.138ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13122025-04-17T00:31:18.138ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13132025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13142025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13152025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13162025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13172025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13182025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13192025-04-17T00:31:18.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13202025-04-17T00:31:18.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13212025-04-17T00:31:18.143ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13222025-04-17T00:31:18.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13232025-04-17T00:31:18.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13242025-04-17T00:31:18.144ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13252025-04-17T00:31:18.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13262025-04-17T00:31:18.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13272025-04-17T00:31:18.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13282025-04-17T00:31:18.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13292025-04-17T00:31:18.146ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13302025-04-17T00:31:18.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13312025-04-17T00:31:18.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13322025-04-17T00:31:18.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13332025-04-17T00:31:18.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13342025-04-17T00:31:18.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
13352025-04-17T00:31:18.151ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13362025-04-17T00:31:18.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13372025-04-17T00:31:18.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
13382025-04-17T00:31:18.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13392025-04-17T00:31:18.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13402025-04-17T00:31:18.154ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13412025-04-17T00:31:18.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13422025-04-17T00:31:18.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13432025-04-17T00:31:18.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13442025-04-17T00:31:18.165ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
13452025-04-17T00:31:18.165ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 203.065505ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
13462025-04-17T00:31:18.165ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13472025-04-17T00:31:18.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13482025-04-17T00:31:18.168ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 206.974033ms iteration = 21 status = 0
13492025-04-17T00:31:18.170ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13502025-04-17T00:31:18.170ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
13512025-04-17T00:31:18.170ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 208.797399ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
13522025-04-17T00:31:18.173ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13532025-04-17T00:31:18.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13542025-04-17T00:31:18.173ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
13552025-04-17T00:31:18.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13562025-04-17T00:31:18.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13572025-04-17T00:31:18.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13582025-04-17T00:31:18.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13592025-04-17T00:31:18.180ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 218.864363ms iteration = 23 status = {"physical_disks_added":0}
13602025-04-17T00:31:18.181ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13612025-04-17T00:31:18.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13622025-04-17T00:31:18.186ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13632025-04-17T00:31:18.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13642025-04-17T00:31:18.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13652025-04-17T00:31:18.186ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
13662025-04-17T00:31:18.186ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 224.962506ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
13672025-04-17T00:31:18.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13682025-04-17T00:31:18.189ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13692025-04-17T00:31:18.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13702025-04-17T00:31:18.190ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
13712025-04-17T00:31:18.190ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 228.794473ms iteration = 5 status = {"blueprint_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","inventory_collection_id":"9ab35570-ac28-4269-bde2-a430166fc41c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
13722025-04-17T00:31:18.191ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13732025-04-17T00:31:18.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13742025-04-17T00:31:18.195ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13752025-04-17T00:31:18.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13762025-04-17T00:31:18.197ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13772025-04-17T00:31:18.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13782025-04-17T00:31:18.198ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
13792025-04-17T00:31:18.198ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
13802025-04-17T00:31:18.198ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
13812025-04-17T00:31:18.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13822025-04-17T00:31:18.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13832025-04-17T00:31:18.200ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 238.437006ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
13842025-04-17T00:31:18.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13852025-04-17T00:31:18.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13862025-04-17T00:31:18.208ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13872025-04-17T00:31:18.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13882025-04-17T00:31:18.211ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13892025-04-17T00:31:18.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13902025-04-17T00:31:18.213ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13912025-04-17T00:31:18.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13922025-04-17T00:31:18.214ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13932025-04-17T00:31:18.214ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13942025-04-17T00:31:18.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13952025-04-17T00:31:18.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13962025-04-17T00:31:18.223ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13972025-04-17T00:31:18.225ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
13982025-04-17T00:31:18.225ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
13992025-04-17T00:31:18.225ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14002025-04-17T00:31:18.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14012025-04-17T00:31:18.226ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14022025-04-17T00:31:18.226ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14032025-04-17T00:31:18.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14042025-04-17T00:31:18.229ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14052025-04-17T00:31:18.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14062025-04-17T00:31:18.234ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14072025-04-17T00:31:18.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14082025-04-17T00:31:18.234ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
14092025-04-17T00:31:18.234ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 272.380521ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
14102025-04-17T00:31:18.239ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
14112025-04-17T00:31:18.239ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
14122025-04-17T00:31:18.239ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-17 00:21:18.231041 UTC dns_group = internal generation = 1
14132025-04-17T00:31:18.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 277.275634ms iteration = 11 status = {"generation":1}
14142025-04-17T00:31:18.275ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
14152025-04-17T00:31:18.275ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 0 version = 2
14162025-04-17T00:31:18.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14172025-04-17T00:31:18.275ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
14182025-04-17T00:31:18.275ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14192025-04-17T00:31:18.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14202025-04-17T00:31:18.276ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "30aaa44b-1bef-4d67-a244-af994e8ca9ca", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
14212025-04-17T00:31:18.276ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14222025-04-17T00:31:18.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14232025-04-17T00:31:18.280ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
14242025-04-17T00:31:18.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14252025-04-17T00:31:18.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14262025-04-17T00:31:18.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14272025-04-17T00:31:18.285ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14282025-04-17T00:31:18.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14292025-04-17T00:31:18.287ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 1 version = 2
14302025-04-17T00:31:18.287ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
14312025-04-17T00:31:18.287ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-17 00:21:19.348689 UTC dns_group = external generation = 2
14322025-04-17T00:31:18.287ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 325.16998ms iteration = 11 status = {"generation":2}
14332025-04-17T00:31:18.288ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342025-04-17T00:31:18.293ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14352025-04-17T00:31:18.293ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14362025-04-17T00:31:18.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14372025-04-17T00:31:18.312ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
14382025-04-17T00:31:18.323ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-17T00:21:15.858908Z, time_modified: 2025-04-17T00:21:15.858908Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }) }
14392025-04-17T00:31:18.323ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 362.038689ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]}},"default_endpoint":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
14402025-04-17T00:31:18.330ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
14412025-04-17T00:31:18.330ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14422025-04-17T00:31:18.330ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2462a8db-b294-49be-8a93-b66584d82fd2", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
14432025-04-17T00:31:18.330ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14442025-04-17T00:31:18.332ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14452025-04-17T00:31:18.349ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14462025-04-17T00:31:18.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14472025-04-17T00:31:18.356ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14482025-04-17T00:31:18.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14492025-04-17T00:31:18.416ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
14502025-04-17T00:31:18.417ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "300f5ee6-7626-431a-a3de-3b06099020ec", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14512025-04-17T00:31:18.417ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14522025-04-17T00:31:18.425ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e7ad00c3-f8f5-4da0-b658-bc41385457cf", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14532025-04-17T00:31:18.425ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14542025-04-17T00:31:18.429ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14552025-04-17T00:31:18.429ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 467.052484ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
14562025-04-17T00:31:18.481ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14572025-04-17T00:31:18.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14582025-04-17T00:31:18.483ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14592025-04-17T00:31:18.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14602025-04-17T00:31:18.485ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 523.845725ms iteration = 21 status = null
14612025-04-17T00:31:18.485ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14622025-04-17T00:31:18.485ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14632025-04-17T00:31:18.486ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
14642025-04-17T00:31:18.486ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14652025-04-17T00:31:18.486ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14662025-04-17T00:31:18.486ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "eee126ce-60db-494f-bcb1-20e718c29d9d", "content-length": "372", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
14672025-04-17T00:31:18.486ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14682025-04-17T00:31:18.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14692025-04-17T00:31:18.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14702025-04-17T00:31:18.487ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
14712025-04-17T00:31:18.488ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14722025-04-17T00:31:18.488ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c727c01f-f204-47c2-a4e3-88ed385c9af4", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14732025-04-17T00:31:18.488ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14742025-04-17T00:31:18.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14752025-04-17T00:31:18.489ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14762025-04-17T00:31:18.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14772025-04-17T00:31:18.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
14782025-04-17T00:31:18.489ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14792025-04-17T00:31:18.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14802025-04-17T00:31:18.490ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "270cec63-0b35-4cfe-82ca-7cc3c9ef733f", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14812025-04-17T00:31:18.490ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14822025-04-17T00:31:18.491ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14832025-04-17T00:31:18.492ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "db0014ae-4240-49ea-b99a-a5e7082c6df2", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14842025-04-17T00:31:18.492ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14852025-04-17T00:31:18.492ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14862025-04-17T00:31:18.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14872025-04-17T00:31:18.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
14882025-04-17T00:31:18.493ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14892025-04-17T00:31:18.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14902025-04-17T00:31:18.495ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14912025-04-17T00:31:18.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14922025-04-17T00:31:18.495ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14932025-04-17T00:31:18.496ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14942025-04-17T00:31:18.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14952025-04-17T00:31:18.496ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 534.598975ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
14962025-04-17T00:31:18.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
14972025-04-17T00:31:18.497ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14982025-04-17T00:31:18.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14992025-04-17T00:31:18.498ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 537.362386ms iteration = 21 status = {}
15002025-04-17T00:31:18.500ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
15012025-04-17T00:31:18.500ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15022025-04-17T00:31:18.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15032025-04-17T00:31:18.502ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "6799b5dc-92d5-4aaf-9adb-0fb9b64fd38f", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15042025-04-17T00:31:18.502ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
15052025-04-17T00:31:18.502ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "542c4f46-741d-4d4e-8dae-462ec1f02a0b", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15062025-04-17T00:31:18.502ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
15072025-04-17T00:31:18.503ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15082025-04-17T00:31:18.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15092025-04-17T00:31:18.504ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 542.577296ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"ca62f5ab-0935-4588-a5c0-db1db0e32680","time":"2025-04-17T00:21:18.520259947Z"},{"operation":"list","target_sled":"ca62f5ab-0935-4588-a5c0-db1db0e32680","time":"2025-04-17T00:21:18.521541380Z"},{"generation":1,"operation":"put_config","target_sled":"ca62f5ab-0935-4588-a5c0-db1db0e32680","time":"2025-04-17T00:26:18.612131653Z"},{"operation":"list","target_sled":"ca62f5ab-0935-4588-a5c0-db1db0e32680","time":"2025-04-17T00:26:18.612593635Z"},{"generation":1,"operation":"put_config","target_sled":"ca62f5ab-0935-4588-a5c0-db1db0e32680","time":"2025-04-17T00:31:18.502315808Z"},{"operation":"list","target_sled":"ca62f5ab-0935-4588-a5c0-db1db0e32680","time":"2025-04-17T00:31:18.502732039Z"}]}
15102025-04-17T00:31:18.505ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15112025-04-17T00:31:18.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15122025-04-17T00:31:18.509ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15132025-04-17T00:31:18.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15142025-04-17T00:31:18.512ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15152025-04-17T00:31:18.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15162025-04-17T00:31:18.514ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15172025-04-17T00:31:18.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15182025-04-17T00:31:18.516ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15192025-04-17T00:31:18.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15202025-04-17T00:31:18.518ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15212025-04-17T00:31:18.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15222025-04-17T00:31:18.522ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15232025-04-17T00:31:18.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15242025-04-17T00:31:18.524ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15252025-04-17T00:31:18.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15262025-04-17T00:31:18.531ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15272025-04-17T00:31:18.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15282025-04-17T00:31:18.534ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15292025-04-17T00:31:18.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15302025-04-17T00:31:18.539ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15312025-04-17T00:31:18.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15322025-04-17T00:31:18.556ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
15332025-04-17T00:31:18.556ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
15342025-04-17T00:31:18.556ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "37d530ef-7a5b-48d6-9eee-f18b6db8aef7", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15352025-04-17T00:31:18.556ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15362025-04-17T00:31:18.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15372025-04-17T00:31:18.557ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 3942c148-d84e-401d-a204-3182b45d5291, kind: Service { id: c4d82657-8205-4ac8-b622-739dd8e6c53f }, name: Name("external-dns-c4d82657-8205-4ac8-b622-739dd8e6c53f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 142])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c1da10dd-286f-48e7-bfa8-ed2e5829d0cb, kind: Service { id: 99145431-02b3-4b6e-aa5d-968f96332449 }, name: Name("external-dns-99145431-02b3-4b6e-aa5d-968f96332449"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 236, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 2675defd-8041-4c37-b3dc-c6b38d20a2ab, kind: Service { id: f40be775-6972-4c7e-b3a1-9a8ec86abe24 }, name: Name("nexus-f40be775-6972-4c7e-b3a1-9a8ec86abe24"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 980b3b66-75dc-42a8-8e05-3181d4b5855e, kind: Service { id: ef9eec85-7cbe-4a5f-89ba-f11462071c47 }, name: Name("nexus-ef9eec85-7cbe-4a5f-89ba-f11462071c47"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 185, 97])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: aca9a30b-f820-4001-ae2d-18cd82c7b824, kind: Service { id: 62e378e4-0af6-4ff9-8300-c58459bfd026 }, name: Name("nexus-62e378e4-0af6-4ff9-8300-c58459bfd026"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
15382025-04-17T00:31:18.557ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15392025-04-17T00:31:18.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15402025-04-17T00:31:18.559ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412025-04-17T00:31:18.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15422025-04-17T00:31:18.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15432025-04-17T00:31:18.561ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
15442025-04-17T00:31:18.561ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15452025-04-17T00:31:18.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15462025-04-17T00:31:18.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15472025-04-17T00:31:18.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15482025-04-17T00:31:18.568ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
15492025-04-17T00:31:18.568ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15502025-04-17T00:31:18.568ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "65975c8f-cd69-43bf-8497-8c3453ff9c82", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15512025-04-17T00:31:18.571ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 609.45092ms iteration = 21 status = {}
15522025-04-17T00:31:18.574ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
15532025-04-17T00:31:18.575ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3942c148-d84e-401d-a204-3182b45d5291, kind: Service { id: c4d82657-8205-4ac8-b622-739dd8e6c53f }, name: Name("external-dns-c4d82657-8205-4ac8-b622-739dd8e6c53f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 142])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c1da10dd-286f-48e7-bfa8-ed2e5829d0cb, kind: Service { id: 99145431-02b3-4b6e-aa5d-968f96332449 }, name: Name("external-dns-99145431-02b3-4b6e-aa5d-968f96332449"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 236, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 2675defd-8041-4c37-b3dc-c6b38d20a2ab, kind: Service { id: f40be775-6972-4c7e-b3a1-9a8ec86abe24 }, name: Name("nexus-f40be775-6972-4c7e-b3a1-9a8ec86abe24"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 980b3b66-75dc-42a8-8e05-3181d4b5855e, kind: Service { id: ef9eec85-7cbe-4a5f-89ba-f11462071c47 }, name: Name("nexus-ef9eec85-7cbe-4a5f-89ba-f11462071c47"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 185, 97])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: aca9a30b-f820-4001-ae2d-18cd82c7b824, kind: Service { id: 62e378e4-0af6-4ff9-8300-c58459bfd026 }, name: Name("nexus-62e378e4-0af6-4ff9-8300-c58459bfd026"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
15542025-04-17T00:31:18.575ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
15552025-04-17T00:31:18.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15562025-04-17T00:31:18.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15572025-04-17T00:31:18.631ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15582025-04-17T00:31:18.634ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
15592025-04-17T00:31:18.635ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15602025-04-17T00:31:18.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15612025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
15622025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
15632025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
15642025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
15652025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15662025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15672025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15682025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15692025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15702025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15712025-04-17T00:31:18.648ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
15722025-04-17T00:31:18.648ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15732025-04-17T00:31:18.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15742025-04-17T00:31:18.651ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
15752025-04-17T00:31:18.651ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15762025-04-17T00:31:18.714ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1323a27-5174-4e35-ac8e-c75d47d9e1c0", "content-length": "691", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15772025-04-17T00:31:18.714ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
15782025-04-17T00:31:18.714ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
15792025-04-17T00:31:18.714ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
15802025-04-17T00:31:18.716ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 754.503924ms iteration = 21 status = {}
15812025-04-17T00:31:18.724ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: ca62f5ab-0935-4588-a5c0-db1db0e32680, time_created: 2025-04-17T00:21:17.626068Z, time_modified: 2025-04-17T00:21:17.626068Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
15822025-04-17T00:31:18.724ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
15832025-04-17T00:31:18.724ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15842025-04-17T00:31:18.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15852025-04-17T00:31:18.726ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15862025-04-17T00:31:18.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15872025-04-17T00:31:18.728ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15882025-04-17T00:31:18.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
15892025-04-17T00:31:18.768ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
15902025-04-17T00:31:18.771ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3cce1236-9624-439b-8416-7ffb42785167", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15912025-04-17T00:31:18.772ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
15922025-04-17T00:31:18.772ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 811.255673ms
15932025-04-17T00:31:18.772ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 811.264573ms iteration = 3 status = {}
15942025-04-17T00:31:18.840ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9b0d6b6c-42b9-4541-bfaf-93bc738aede7", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15952025-04-17T00:31:18.840ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
15962025-04-17T00:31:18.840ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15972025-04-17T00:31:18.846ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0d0f5af-625c-4acc-988f-aa0e79067977", "content-length": "682", "date": "Thu, 17 Apr 2025 00:31:18 GMT"} })
15982025-04-17T00:31:18.846ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15992025-04-17T00:31:18.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16002025-04-17T00:31:18.851ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16012025-04-17T00:31:18.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16022025-04-17T00:31:18.854ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16032025-04-17T00:31:18.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16042025-04-17T00:31:18.857ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16052025-04-17T00:31:18.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16062025-04-17T00:31:18.865ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 903.331992ms iteration = 21 status = {}
16072025-04-17T00:31:18.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0a74370c-9490-4918-8279-17bc53e23dc9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
16082025-04-17T00:31:18.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0a74370c-9490-4918-8279-17bc53e23dc9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
16092025-04-17T00:31:18.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2342 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0a74370c-9490-4918-8279-17bc53e23dc9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
16102025-04-17T00:31:19.271ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
16112025-04-17T00:31:19.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16122025-04-17T00:31:19.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16132025-04-17T00:31:19.274ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16142025-04-17T00:31:19.276ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
16152025-04-17T00:31:19.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16162025-04-17T00:31:19.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16172025-04-17T00:31:19.280ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(6a3acd02-e775-41cc-be66-601d0db26c66, 0), (5c0448fe-041a-4fc4-be73-94e9f775eccc, 0), (9ab35570-ac28-4269-bde2-a430166fc41c, 0), (f5e79b57-132e-4ef7-94a4-731b0d795801, 0)] collection_id = 6a3acd02-e775-41cc-be66-601d0db26c66
16182025-04-17T00:31:19.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16192025-04-17T00:31:19.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16202025-04-17T00:31:19.376ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 6a3acd02-e775-41cc-be66-601d0db26c66 file = nexus/db-queries/src/db/datastore/inventory.rs:1446 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
16212025-04-17T00:31:19.376ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16222025-04-17T00:31:19.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16232025-04-17T00:31:19.379ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(5c0448fe-041a-4fc4-be73-94e9f775eccc, 0), (9ab35570-ac28-4269-bde2-a430166fc41c, 0), (f5e79b57-132e-4ef7-94a4-731b0d795801, 0), (7473c0de-f68b-4446-a433-37922d445d78, 0)] collection_id = 5c0448fe-041a-4fc4-be73-94e9f775eccc
16242025-04-17T00:31:19.379ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16252025-04-17T00:31:19.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16262025-04-17T00:31:19.434ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 5c0448fe-041a-4fc4-be73-94e9f775eccc file = nexus/db-queries/src/db/datastore/inventory.rs:1446 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
16272025-04-17T00:31:19.434ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16282025-04-17T00:31:19.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16292025-04-17T00:31:19.441ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(9ab35570-ac28-4269-bde2-a430166fc41c, 0), (f5e79b57-132e-4ef7-94a4-731b0d795801, 0), (7473c0de-f68b-4446-a433-37922d445d78, 0)]
16302025-04-17T00:31:19.442ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1805, tv_nsec: 113331315 } })
16312025-04-17T00:31:19.474ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
16322025-04-17T00:31:19.474ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
16332025-04-17T00:31:19.474ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
16342025-04-17T00:31:19.474ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "07ec6eb9-416a-4e16-9a97-246599e5ac60", "content-length": "476", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16352025-04-17T00:31:19.475ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
16362025-04-17T00:31:19.475ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "71f79548-c91e-4ec6-af3f-c2b31639ea46", "content-length": "734", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16372025-04-17T00:31:19.475ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
16382025-04-17T00:31:19.475ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb150ebd-5fa5-4d53-8fe4-90bc4d1c366b", "content-length": "111", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16392025-04-17T00:31:19.475ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
16402025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb24b398-ddb4-4de5-84bf-b95ed6082a07", "content-length": "111", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16412025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
16422025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8c0173f-5439-4e6c-ba16-005c22e3a7ab", "content-length": "167", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16432025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
16442025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff634eb8-4561-46df-8269-5b54ab62faea", "content-length": "167", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16452025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
16462025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85d2ff84-eb46-40a0-bd7d-cd376c92aae6", "content-length": "175", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16472025-04-17T00:31:19.476ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
16482025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "755783d8-655c-4efd-b014-76bfd84ae42e", "content-length": "175", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16492025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
16502025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eb2e46d-0924-4403-8dae-adad27af2c86", "content-length": "702", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16512025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
16522025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7935fb35-dc28-4b41-9830-0bba1076e3f0", "content-length": "727", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16532025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
16542025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "46426708-082d-4189-9e07-42c7d12c89fe", "content-length": "729", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16552025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
16562025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0aa216aa-ef90-4734-9ede-5628dd24d84e", "content-length": "728", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16572025-04-17T00:31:19.477ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
16582025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97e79cfd-62e4-4406-945c-f7228d4f9fbd", "content-length": "722", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16592025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
16602025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2777bd8f-c0fa-4bd8-9c4c-53013ac115ba", "content-length": "109", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16612025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
16622025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf7887c0-6d11-41ae-8d77-79f21ae28470", "content-length": "109", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16632025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
16642025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "08da0f99-1ef8-4725-8c79-e265f2efb52a", "content-length": "169", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16652025-04-17T00:31:19.478ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
16662025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dd94eb8-7e78-41cc-a9dc-57f5bbb0f93a", "content-length": "169", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16672025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
16682025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eccf176-2ea1-425f-a5dd-5472d77cf493", "content-length": "178", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16692025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
16702025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8bc2eab-2b44-485e-93f9-6d41c6523c3e", "content-length": "178", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16712025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
16722025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfc46f79-81dc-4acb-909e-c494feff01b6", "content-length": "702", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16732025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16742025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c40ca56-f22c-4085-b8c0-7a25b5484f25", "content-length": "727", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16752025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16762025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc9b1a02-6d48-45b5-9322-a4375e4489c4", "content-length": "729", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16772025-04-17T00:31:19.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16782025-04-17T00:31:19.480ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6daa894-a52c-42e7-88e4-a67b2f405295", "content-length": "728", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16792025-04-17T00:31:19.480ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16802025-04-17T00:31:19.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16812025-04-17T00:31:19.481ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16822025-04-17T00:31:19.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16832025-04-17T00:31:19.483ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16842025-04-17T00:31:19.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16852025-04-17T00:31:19.484ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16862025-04-17T00:31:19.486ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16872025-04-17T00:31:19.486ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16882025-04-17T00:31:19.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16892025-04-17T00:31:19.521ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
16902025-04-17T00:31:19.521ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
16912025-04-17T00:31:19.719ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "208aa5fc-5cab-42b1-bfa0-5cc0b69852d0", "content-length": "31974", "date": "Thu, 17 Apr 2025 00:31:19 GMT"} })
16922025-04-17T00:31:19.721ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16932025-04-17T00:31:19.721ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16942025-04-17T00:31:19.721ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
16952025-04-17T00:31:19.722ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16962025-04-17T00:31:19.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16972025-04-17T00:31:19.724ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16982025-04-17T00:31:19.726ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
16992025-04-17T00:31:19.726ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17002025-04-17T00:31:19.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
17012025-04-17T00:31:19.864ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = cf0a587e-ab39-4be2-85fd-7d34b5e70bc8 file = nexus/db-queries/src/db/datastore/inventory.rs:1009
17022025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = cf0a587e-ab39-4be2-85fd-7d34b5e70bc8 time_started = 2025-04-17 00:31:19.474495 UTC
17032025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
17042025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17052025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 592.526985ms iteration = 3 status = {"collection_id":"cf0a587e-ab39-4be2-85fd-7d34b5e70bc8","time_done":"2025-04-17 00:31:19.721297 UTC","time_started":"2025-04-17 00:31:19.474495 UTC"}
17062025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
17072025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17082025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17092025-04-17T00:31:19.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17102025-04-17T00:31:19.866ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17112025-04-17T00:31:19.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17122025-04-17T00:31:19.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17132025-04-17T00:31:19.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17142025-04-17T00:31:19.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17152025-04-17T00:31:19.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
17162025-04-17T00:31:19.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17172025-04-17T00:31:19.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17182025-04-17T00:31:19.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17192025-04-17T00:31:19.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17202025-04-17T00:31:19.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17212025-04-17T00:31:19.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17222025-04-17T00:31:19.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17232025-04-17T00:31:19.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17242025-04-17T00:31:19.875ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17252025-04-17T00:31:19.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17262025-04-17T00:31:19.887ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 23.239456ms iteration = 24 status = {"physical_disks_added":0}
17272025-04-17T00:31:19.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17282025-04-17T00:31:19.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17292025-04-17T00:31:19.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17302025-04-17T00:31:19.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17312025-04-17T00:31:19.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17322025-04-17T00:31:19.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17332025-04-17T00:31:19.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17342025-04-17T00:31:19.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17352025-04-17T00:31:19.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17362025-04-17T00:31:19.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17372025-04-17T00:31:19.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17382025-04-17T00:31:19.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17392025-04-17T00:31:19.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17402025-04-17T00:31:19.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17412025-04-17T00:31:19.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17422025-04-17T00:31:19.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17432025-04-17T00:31:19.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17442025-04-17T00:31:19.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17452025-04-17T00:31:19.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17462025-04-17T00:31:19.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17472025-04-17T00:31:19.922ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
17482025-04-17T00:31:19.922ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 58.296058ms iteration = 6 status = {"blueprint_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","inventory_collection_id":"cf0a587e-ab39-4be2-85fd-7d34b5e70bc8","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
17492025-04-17T00:31:19.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 55c98eef-91af-40cf-9f61-4d83fff633eb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17502025-04-17T00:31:19.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 55c98eef-91af-40cf-9f61-4d83fff633eb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17512025-04-17T00:31:19.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2015 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 55c98eef-91af-40cf-9f61-4d83fff633eb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17522025-04-17T00:31:20.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 31b172c2-2964-4f07-a1cd-a4d16af79b0a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17532025-04-17T00:31:20.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 31b172c2-2964-4f07-a1cd-a4d16af79b0a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17542025-04-17T00:31:20.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2569 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 31b172c2-2964-4f07-a1cd-a4d16af79b0a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17552025-04-17T00:31:21.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 460 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 26d1828c-80ec-45a4-a17c-8f3eb790ff5b response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
17562025-04-17T00:31:21.989ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3577328c-f2a9-46b9-bf9e-d06a5c0657d0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17572025-04-17T00:31:21.989ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3577328c-f2a9-46b9-bf9e-d06a5c0657d0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17582025-04-17T00:31:21.991ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3577328c-f2a9-46b9-bf9e-d06a5c0657d0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17592025-04-17T00:31:23.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1df232d1-7abc-4513-b944-b8dc3701cc62 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17602025-04-17T00:31:23.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1df232d1-7abc-4513-b944-b8dc3701cc62 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17612025-04-17T00:31:23.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1938 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1df232d1-7abc-4513-b944-b8dc3701cc62 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17622025-04-17T00:31:24.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 511e266a-b4aa-4bd8-9c0d-8b0ac45e09d4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17632025-04-17T00:31:24.033ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 511e266a-b4aa-4bd8-9c0d-8b0ac45e09d4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17642025-04-17T00:31:24.035ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2002 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 511e266a-b4aa-4bd8-9c0d-8b0ac45e09d4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17652025-04-17T00:31:25.057ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e3ed7fe7-f551-4910-80db-ca4ea2a3f71d roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17662025-04-17T00:31:25.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e3ed7fe7-f551-4910-80db-ca4ea2a3f71d resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17672025-04-17T00:31:25.059ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2674 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e3ed7fe7-f551-4910-80db-ca4ea2a3f71d response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17682025-04-17T00:31:26.079ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9b1658ad-b9ad-4113-92b5-ce0f981180b2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17692025-04-17T00:31:26.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9b1658ad-b9ad-4113-92b5-ce0f981180b2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17702025-04-17T00:31:26.081ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2452 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9b1658ad-b9ad-4113-92b5-ce0f981180b2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17712025-04-17T00:31:27.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4272ed1e-fb7a-4180-b12e-fd87e26771ed roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17722025-04-17T00:31:27.100ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4272ed1e-fb7a-4180-b12e-fd87e26771ed resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17732025-04-17T00:31:27.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1973 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4272ed1e-fb7a-4180-b12e-fd87e26771ed response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17742025-04-17T00:31:27.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
17752025-04-17T00:31:27.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17762025-04-17T00:31:27.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
17772025-04-17T00:31:27.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17782025-04-17T00:31:27.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17792025-04-17T00:31:27.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17802025-04-17T00:31:27.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17812025-04-17T00:31:27.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17822025-04-17T00:31:27.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17832025-04-17T00:31:27.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17842025-04-17T00:31:27.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17852025-04-17T00:31:27.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17862025-04-17T00:31:27.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17872025-04-17T00:31:27.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17882025-04-17T00:31:27.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17892025-04-17T00:31:27.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
17902025-04-17T00:31:27.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17912025-04-17T00:31:27.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17922025-04-17T00:31:27.978ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
17932025-04-17T00:31:27.978ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.671345ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
17942025-04-17T00:31:27.992ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.647367ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17952025-04-17T00:31:28.121ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d873733d-f6ed-4e92-8e86-7a83d46d6988 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17962025-04-17T00:31:28.122ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d873733d-f6ed-4e92-8e86-7a83d46d6988 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17972025-04-17T00:31:28.123ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1934 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d873733d-f6ed-4e92-8e86-7a83d46d6988 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17982025-04-17T00:31:29.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fca30528-276f-4db8-b4cb-1628a79e5fb8 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
17992025-04-17T00:31:29.144ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fca30528-276f-4db8-b4cb-1628a79e5fb8 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18002025-04-17T00:31:29.146ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2636 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fca30528-276f-4db8-b4cb-1628a79e5fb8 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18012025-04-17T00:31:30.165ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b49229d1-0470-452a-9036-69f1729c6094 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18022025-04-17T00:31:30.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b49229d1-0470-452a-9036-69f1729c6094 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18032025-04-17T00:31:30.167ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1977 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b49229d1-0470-452a-9036-69f1729c6094 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18042025-04-17T00:31:31.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e5ce8217-401e-4192-8203-dd6a9e0cbc95 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18052025-04-17T00:31:31.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e5ce8217-401e-4192-8203-dd6a9e0cbc95 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18062025-04-17T00:31:31.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2721 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e5ce8217-401e-4192-8203-dd6a9e0cbc95 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18072025-04-17T00:31:31.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 635 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 1d25c938-2f32-4df2-83b6-08a342ced158 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
18082025-04-17T00:31:32.210ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 275bf154-c29d-449b-b4cc-45e6a628a8d6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18092025-04-17T00:31:32.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 275bf154-c29d-449b-b4cc-45e6a628a8d6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18102025-04-17T00:31:32.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2174 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 275bf154-c29d-449b-b4cc-45e6a628a8d6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18112025-04-17T00:31:33.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ab3961dd-9600-4449-9279-81de0e97c1ce roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18122025-04-17T00:31:33.233ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ab3961dd-9600-4449-9279-81de0e97c1ce resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18132025-04-17T00:31:33.235ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ab3961dd-9600-4449-9279-81de0e97c1ce response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18142025-04-17T00:31:34.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 603fdb59-8238-4324-9745-b7fdebc9210c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18152025-04-17T00:31:34.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 603fdb59-8238-4324-9745-b7fdebc9210c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18162025-04-17T00:31:34.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1821 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 603fdb59-8238-4324-9745-b7fdebc9210c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18172025-04-17T00:31:35.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8b1d02f0-ef21-4f7e-95f4-eded421b6ef8 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18182025-04-17T00:31:35.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8b1d02f0-ef21-4f7e-95f4-eded421b6ef8 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18192025-04-17T00:31:35.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2112 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8b1d02f0-ef21-4f7e-95f4-eded421b6ef8 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18202025-04-17T00:31:36.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f0cb1ed3-ed71-4cb8-8d62-06f846b3db5f roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18212025-04-17T00:31:36.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f0cb1ed3-ed71-4cb8-8d62-06f846b3db5f resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18222025-04-17T00:31:36.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2326 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f0cb1ed3-ed71-4cb8-8d62-06f846b3db5f response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18232025-04-17T00:31:37.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c2424c04-10dd-439c-9b75-e3c79e0cfab4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18242025-04-17T00:31:37.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c2424c04-10dd-439c-9b75-e3c79e0cfab4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18252025-04-17T00:31:37.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1791 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c2424c04-10dd-439c-9b75-e3c79e0cfab4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18262025-04-17T00:31:37.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
18272025-04-17T00:31:37.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
18282025-04-17T00:31:37.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18292025-04-17T00:31:37.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18302025-04-17T00:31:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18312025-04-17T00:31:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18322025-04-17T00:31:37.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18332025-04-17T00:31:37.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18342025-04-17T00:31:37.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18352025-04-17T00:31:37.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18362025-04-17T00:31:37.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18372025-04-17T00:31:37.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18382025-04-17T00:31:37.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18392025-04-17T00:31:37.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18402025-04-17T00:31:37.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18412025-04-17T00:31:37.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
18422025-04-17T00:31:37.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18432025-04-17T00:31:37.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18442025-04-17T00:31:37.981ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
18452025-04-17T00:31:37.981ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.27101ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
18462025-04-17T00:31:37.992ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.808761ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18472025-04-17T00:31:38.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cff6d46c-4a76-4a42-a08c-0af710895528 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18482025-04-17T00:31:38.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cff6d46c-4a76-4a42-a08c-0af710895528 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18492025-04-17T00:31:38.341ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1805 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cff6d46c-4a76-4a42-a08c-0af710895528 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18502025-04-17T00:31:39.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7bb7311f-30ac-48f8-8e19-ae14328ce072 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18512025-04-17T00:31:39.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7bb7311f-30ac-48f8-8e19-ae14328ce072 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18522025-04-17T00:31:39.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2243 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7bb7311f-30ac-48f8-8e19-ae14328ce072 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18532025-04-17T00:31:40.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 904991bb-9c7a-430b-aea2-2be58fae4743 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18542025-04-17T00:31:40.384ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 904991bb-9c7a-430b-aea2-2be58fae4743 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18552025-04-17T00:31:40.386ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2232 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 904991bb-9c7a-430b-aea2-2be58fae4743 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18562025-04-17T00:31:41.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 302c59b9-632c-46ab-8fe4-2bccfb50f15e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18572025-04-17T00:31:41.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 302c59b9-632c-46ab-8fe4-2bccfb50f15e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18582025-04-17T00:31:41.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2077 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 302c59b9-632c-46ab-8fe4-2bccfb50f15e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18592025-04-17T00:31:41.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 550 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = f4024fc3-262f-4585-ac9f-1efc2ace5de1 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
18602025-04-17T00:31:42.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90b9ddee-5014-4e6d-b878-a516642c25ac roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18612025-04-17T00:31:42.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90b9ddee-5014-4e6d-b878-a516642c25ac resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18622025-04-17T00:31:42.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2191 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 90b9ddee-5014-4e6d-b878-a516642c25ac response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18632025-04-17T00:31:43.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0ef7d1bb-f072-4b41-9416-87d786cbe5b3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18642025-04-17T00:31:43.450ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0ef7d1bb-f072-4b41-9416-87d786cbe5b3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18652025-04-17T00:31:43.452ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1989 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0ef7d1bb-f072-4b41-9416-87d786cbe5b3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18662025-04-17T00:31:44.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 14c52f28-5d1a-4baf-aa0f-dfa8bda27b9a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18672025-04-17T00:31:44.472ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 14c52f28-5d1a-4baf-aa0f-dfa8bda27b9a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18682025-04-17T00:31:44.474ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2241 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 14c52f28-5d1a-4baf-aa0f-dfa8bda27b9a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18692025-04-17T00:31:45.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bc5dbf09-89d8-4e55-a479-be551558e0d8 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18702025-04-17T00:31:45.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bc5dbf09-89d8-4e55-a479-be551558e0d8 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18712025-04-17T00:31:45.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2651 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bc5dbf09-89d8-4e55-a479-be551558e0d8 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18722025-04-17T00:31:46.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ac50c261-c5a7-41bd-9da9-135f52b5867e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18732025-04-17T00:31:46.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ac50c261-c5a7-41bd-9da9-135f52b5867e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18742025-04-17T00:31:46.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2087 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ac50c261-c5a7-41bd-9da9-135f52b5867e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18752025-04-17T00:31:47.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 15b4e2de-0a9d-4eb6-9ed4-4727fb92716a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18762025-04-17T00:31:47.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 15b4e2de-0a9d-4eb6-9ed4-4727fb92716a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18772025-04-17T00:31:47.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2612 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 15b4e2de-0a9d-4eb6-9ed4-4727fb92716a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
18782025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
18792025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18802025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
18812025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18822025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
18832025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18842025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
18852025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
18862025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18872025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18882025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
18892025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
18902025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
18912025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
18922025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18932025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
18942025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18952025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
18962025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
18972025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
18982025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18992025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
19002025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19012025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
19022025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19032025-04-17T00:31:47.960ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
19042025-04-17T00:31:47.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19052025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19062025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
19072025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
19082025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
19092025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19102025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
19112025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
19122025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19132025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19142025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
19152025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19162025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
19172025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19182025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19192025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19202025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19212025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19222025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19232025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19242025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19252025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
19262025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19272025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
19282025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19292025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19302025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19312025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19322025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19332025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19342025-04-17T00:31:47.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19352025-04-17T00:31:47.963ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1833, tv_nsec: 634718201 } })
19362025-04-17T00:31:47.964ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
19372025-04-17T00:31:47.964ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19382025-04-17T00:31:47.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19392025-04-17T00:31:47.964ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1833, tv_nsec: 635567502 } })
19402025-04-17T00:31:47.964ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19412025-04-17T00:31:47.964ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
19422025-04-17T00:31:47.965ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.002101ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19432025-04-17T00:31:47.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
19442025-04-17T00:31:47.966ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19452025-04-17T00:31:47.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19462025-04-17T00:31:47.968ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19472025-04-17T00:31:47.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19482025-04-17T00:31:47.971ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19492025-04-17T00:31:47.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
19502025-04-17T00:31:47.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19512025-04-17T00:31:47.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19522025-04-17T00:31:47.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
19532025-04-17T00:31:47.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19542025-04-17T00:31:47.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19552025-04-17T00:31:47.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19562025-04-17T00:31:47.974ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19572025-04-17T00:31:47.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19582025-04-17T00:31:47.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19592025-04-17T00:31:47.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19602025-04-17T00:31:47.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19612025-04-17T00:31:47.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19622025-04-17T00:31:47.982ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19632025-04-17T00:31:47.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
19642025-04-17T00:31:47.985ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19652025-04-17T00:31:47.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19662025-04-17T00:31:47.986ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1833, tv_nsec: 657000912 } })
19672025-04-17T00:31:47.986ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
19682025-04-17T00:31:47.988ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19692025-04-17T00:31:47.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19702025-04-17T00:31:47.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19712025-04-17T00:31:47.990ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19722025-04-17T00:31:47.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19732025-04-17T00:31:47.991ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19742025-04-17T00:31:47.991ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19752025-04-17T00:31:47.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19762025-04-17T00:31:47.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19772025-04-17T00:31:47.996ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
19782025-04-17T00:31:47.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19792025-04-17T00:31:47.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19802025-04-17T00:31:48.001ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19812025-04-17T00:31:48.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19822025-04-17T00:31:48.023ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 62.084124ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
19832025-04-17T00:31:48.025ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
19842025-04-17T00:31:48.026ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 65.10713ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
19852025-04-17T00:31:48.029ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19862025-04-17T00:31:48.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19872025-04-17T00:31:48.031ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19882025-04-17T00:31:48.031ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19892025-04-17T00:31:48.031ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
19902025-04-17T00:31:48.031ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19912025-04-17T00:31:48.031ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 70.696491ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19922025-04-17T00:31:48.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19932025-04-17T00:31:48.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19942025-04-17T00:31:48.034ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19952025-04-17T00:31:48.034ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
19962025-04-17T00:31:48.034ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19972025-04-17T00:31:48.034ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19982025-04-17T00:31:48.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19992025-04-17T00:31:48.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20002025-04-17T00:31:48.035ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20012025-04-17T00:31:48.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20022025-04-17T00:31:48.036ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 75.582253ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
20032025-04-17T00:31:48.037ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20042025-04-17T00:31:48.037ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20052025-04-17T00:31:48.037ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20062025-04-17T00:31:48.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20072025-04-17T00:31:48.038ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 77.873581ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
20082025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20092025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20102025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20112025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20122025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20132025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20142025-04-17T00:31:48.040ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20152025-04-17T00:31:48.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20162025-04-17T00:31:48.042ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20172025-04-17T00:31:48.043ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20182025-04-17T00:31:48.043ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.642602ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20192025-04-17T00:31:48.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20202025-04-17T00:31:48.045ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20212025-04-17T00:31:48.045ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 84.361585ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20222025-04-17T00:31:48.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
20232025-04-17T00:31:48.045ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20242025-04-17T00:31:48.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20252025-04-17T00:31:48.049ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 88.09982ms iteration = 22 status = 0
20262025-04-17T00:31:48.052ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
20272025-04-17T00:31:48.052ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20282025-04-17T00:31:48.052ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "daabe447-7dcc-4161-b0b0-306011db225f", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
20292025-04-17T00:31:48.052ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20302025-04-17T00:31:48.054ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20312025-04-17T00:31:48.054ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20322025-04-17T00:31:48.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20332025-04-17T00:31:48.056ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20342025-04-17T00:31:48.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20352025-04-17T00:31:48.063ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
20362025-04-17T00:31:48.063ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20372025-04-17T00:31:48.063ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "267785bb-dfa5-4fb8-a974-18b91397a0c3", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
20382025-04-17T00:31:48.063ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20392025-04-17T00:31:48.069ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
20402025-04-17T00:31:48.080ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
20412025-04-17T00:31:48.080ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20422025-04-17T00:31:48.081ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a8fc865-721c-477c-9d7d-9dfdcdceb7ae", "content-length": "26", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
20432025-04-17T00:31:48.081ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20442025-04-17T00:31:48.081ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 121.031389ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20452025-04-17T00:31:48.082ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20462025-04-17T00:31:48.083ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20472025-04-17T00:31:48.083ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
20482025-04-17T00:31:48.083ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 122.639069ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
20492025-04-17T00:31:48.086ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 125.766488ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
20502025-04-17T00:31:48.088ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
20512025-04-17T00:31:48.088ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 127.83938ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
20522025-04-17T00:31:48.131ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20532025-04-17T00:31:48.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20542025-04-17T00:31:48.133ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20552025-04-17T00:31:48.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20562025-04-17T00:31:48.181ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 220.518375ms iteration = 22 status = {}
20572025-04-17T00:31:48.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20582025-04-17T00:31:48.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20592025-04-17T00:31:48.212ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 251.281139ms iteration = 25 status = {"physical_disks_added":0}
20602025-04-17T00:31:48.234ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
20612025-04-17T00:31:48.234ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20622025-04-17T00:31:48.234ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "16062d5b-ce75-4d54-b79c-ad961b521edb", "content-length": "372", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
20632025-04-17T00:31:48.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20642025-04-17T00:31:48.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20652025-04-17T00:31:48.237ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 276.600607ms iteration = 22 status = null
20662025-04-17T00:31:48.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20672025-04-17T00:31:48.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20682025-04-17T00:31:48.241ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20692025-04-17T00:31:48.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20702025-04-17T00:31:48.243ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20712025-04-17T00:31:48.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20722025-04-17T00:31:48.245ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20732025-04-17T00:31:48.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20742025-04-17T00:31:48.282ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20752025-04-17T00:31:48.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20762025-04-17T00:31:48.285ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
20772025-04-17T00:31:48.285ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20782025-04-17T00:31:48.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20792025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
20802025-04-17T00:31:48.299ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
20812025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
20822025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
20832025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
20842025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
20852025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20862025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20872025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20882025-04-17T00:31:48.299ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20892025-04-17T00:31:48.300ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20902025-04-17T00:31:48.300ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20912025-04-17T00:31:48.300ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
20922025-04-17T00:31:48.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20932025-04-17T00:31:48.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20942025-04-17T00:31:48.302ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
20952025-04-17T00:31:48.302ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20962025-04-17T00:31:48.305ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
20972025-04-17T00:31:48.410ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "69bcbe3c-64f0-4fc8-9f28-8896f16fc27e", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
20982025-04-17T00:31:48.410ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20992025-04-17T00:31:48.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21002025-04-17T00:31:48.413ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21012025-04-17T00:31:48.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21022025-04-17T00:31:48.414ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f9a5f4c-a485-482e-8d33-9975d4201852", "content-length": "691", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
21032025-04-17T00:31:48.414ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
21042025-04-17T00:31:48.414ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
21052025-04-17T00:31:48.414ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
21062025-04-17T00:31:48.416ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 455.78509ms iteration = 22 status = {}
21072025-04-17T00:31:48.416ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21082025-04-17T00:31:48.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21092025-04-17T00:31:48.419ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
21102025-04-17T00:31:48.419ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21112025-04-17T00:31:48.420ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c2f77c70-b891-444b-a747-96a2e025ea77", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
21122025-04-17T00:31:48.421ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 460.469928ms iteration = 22 status = {}
21132025-04-17T00:31:48.506ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bf8d214a-97ef-4a7a-87d8-15fcecb82ce1", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
21142025-04-17T00:31:48.506ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
21152025-04-17T00:31:48.506ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
21162025-04-17T00:31:48.506ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bbe159b-323b-45e7-872f-510966e9836b", "content-length": "682", "date": "Thu, 17 Apr 2025 00:31:48 GMT"} })
21172025-04-17T00:31:48.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21182025-04-17T00:31:48.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21192025-04-17T00:31:48.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21202025-04-17T00:31:48.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21212025-04-17T00:31:48.513ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21222025-04-17T00:31:48.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21232025-04-17T00:31:48.515ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21242025-04-17T00:31:48.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21252025-04-17T00:31:48.522ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 561.642337ms iteration = 22 status = {}
21262025-04-17T00:31:48.610ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 34b8a2e6-ca77-47ad-bb62-fbdf5cee6ab3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21272025-04-17T00:31:48.610ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 34b8a2e6-ca77-47ad-bb62-fbdf5cee6ab3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21282025-04-17T00:31:48.612ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1975 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 34b8a2e6-ca77-47ad-bb62-fbdf5cee6ab3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21292025-04-17T00:31:49.631ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5be6531e-4c73-4669-843a-aa86ddd920e4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21302025-04-17T00:31:49.631ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5be6531e-4c73-4669-843a-aa86ddd920e4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21312025-04-17T00:31:49.633ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2363 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5be6531e-4c73-4669-843a-aa86ddd920e4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21322025-04-17T00:31:50.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 73e6ce47-d5e1-44a3-8a10-af1a8f0130d9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21332025-04-17T00:31:50.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 73e6ce47-d5e1-44a3-8a10-af1a8f0130d9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21342025-04-17T00:31:50.656ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2502 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 73e6ce47-d5e1-44a3-8a10-af1a8f0130d9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21352025-04-17T00:31:51.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 584 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 329006b3-a9c5-4db7-bd9b-1abb64d25690 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
21362025-04-17T00:31:51.676ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ce21f25b-16df-4630-98cd-4651ab744c01 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21372025-04-17T00:31:51.676ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ce21f25b-16df-4630-98cd-4651ab744c01 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21382025-04-17T00:31:51.678ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ce21f25b-16df-4630-98cd-4651ab744c01 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21392025-04-17T00:31:52.697ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8abe45e4-5cec-4549-af8b-d70a78da02cf roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21402025-04-17T00:31:52.697ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8abe45e4-5cec-4549-af8b-d70a78da02cf resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21412025-04-17T00:31:52.700ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2866 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8abe45e4-5cec-4549-af8b-d70a78da02cf response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21422025-04-17T00:31:53.719ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 46473181-ca08-4624-92eb-910cee2470e9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21432025-04-17T00:31:53.719ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 46473181-ca08-4624-92eb-910cee2470e9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21442025-04-17T00:31:53.720ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1685 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 46473181-ca08-4624-92eb-910cee2470e9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21452025-04-17T00:31:54.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6ce14ce2-47f1-478e-a683-a2d52954dede roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21462025-04-17T00:31:54.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6ce14ce2-47f1-478e-a683-a2d52954dede resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21472025-04-17T00:31:54.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2555 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6ce14ce2-47f1-478e-a683-a2d52954dede response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21482025-04-17T00:31:55.762ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eb2661f6-7e20-4d76-998a-66aaae2f0dee roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21492025-04-17T00:31:55.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eb2661f6-7e20-4d76-998a-66aaae2f0dee resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21502025-04-17T00:31:55.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2584 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eb2661f6-7e20-4d76-998a-66aaae2f0dee response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21512025-04-17T00:31:56.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f62bf34d-33c0-44b9-a6d7-7170cfe860bb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21522025-04-17T00:31:56.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f62bf34d-33c0-44b9-a6d7-7170cfe860bb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21532025-04-17T00:31:56.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f62bf34d-33c0-44b9-a6d7-7170cfe860bb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21542025-04-17T00:31:57.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 786e8ae3-3014-4e4b-8382-6f719ac5b2ee roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21552025-04-17T00:31:57.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 786e8ae3-3014-4e4b-8382-6f719ac5b2ee resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21562025-04-17T00:31:57.806ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2076 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 786e8ae3-3014-4e4b-8382-6f719ac5b2ee response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21572025-04-17T00:31:57.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
21582025-04-17T00:31:57.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
21592025-04-17T00:31:57.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21602025-04-17T00:31:57.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21612025-04-17T00:31:57.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21622025-04-17T00:31:57.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21632025-04-17T00:31:57.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21642025-04-17T00:31:57.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21652025-04-17T00:31:57.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21662025-04-17T00:31:57.965ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.122674ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21672025-04-17T00:31:57.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21682025-04-17T00:31:57.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21692025-04-17T00:31:57.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21702025-04-17T00:31:57.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21712025-04-17T00:31:57.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21722025-04-17T00:31:57.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21732025-04-17T00:31:57.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
21742025-04-17T00:31:57.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21752025-04-17T00:31:57.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21762025-04-17T00:31:57.978ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
21772025-04-17T00:31:57.978ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.759568ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
21782025-04-17T00:31:58.825ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 244022d7-210a-407f-adb6-dc8691c471d5 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21792025-04-17T00:31:58.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 244022d7-210a-407f-adb6-dc8691c471d5 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21802025-04-17T00:31:58.827ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2374 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 244022d7-210a-407f-adb6-dc8691c471d5 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21812025-04-17T00:31:59.846ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1b3119ea-faa3-4009-b0b1-91dbaa4563df roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21822025-04-17T00:31:59.846ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1b3119ea-faa3-4009-b0b1-91dbaa4563df resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21832025-04-17T00:31:59.848ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2254 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1b3119ea-faa3-4009-b0b1-91dbaa4563df response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21842025-04-17T00:32:00.868ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3d62f20b-1ee8-40ce-8d16-3552adca8be0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21852025-04-17T00:32:00.869ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3d62f20b-1ee8-40ce-8d16-3552adca8be0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21862025-04-17T00:32:00.871ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3d62f20b-1ee8-40ce-8d16-3552adca8be0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21872025-04-17T00:32:01.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 480 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = fc9af863-93f6-4737-940c-b3d507afaa77 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
21882025-04-17T00:32:01.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 26c28b76-f3f2-4127-b8ad-a9a4c54635f9 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21892025-04-17T00:32:01.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 26c28b76-f3f2-4127-b8ad-a9a4c54635f9 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21902025-04-17T00:32:01.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1816 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 26c28b76-f3f2-4127-b8ad-a9a4c54635f9 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21912025-04-17T00:32:02.912ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3a91dfe1-b873-4c5b-bc99-2443caf73259 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21922025-04-17T00:32:02.912ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3a91dfe1-b873-4c5b-bc99-2443caf73259 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21932025-04-17T00:32:02.914ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2111 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3a91dfe1-b873-4c5b-bc99-2443caf73259 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21942025-04-17T00:32:03.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b3f3d280-c8ab-4079-b35f-627131850618 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21952025-04-17T00:32:03.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b3f3d280-c8ab-4079-b35f-627131850618 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21962025-04-17T00:32:03.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2475 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b3f3d280-c8ab-4079-b35f-627131850618 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21972025-04-17T00:32:04.959ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a00b42bb-462b-49e0-8dc9-bf22b71ccbe4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21982025-04-17T00:32:04.959ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a00b42bb-462b-49e0-8dc9-bf22b71ccbe4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
21992025-04-17T00:32:04.961ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2505 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a00b42bb-462b-49e0-8dc9-bf22b71ccbe4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22002025-04-17T00:32:05.981ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 79c46b53-bd14-4387-9364-49794134943b roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22012025-04-17T00:32:05.982ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 79c46b53-bd14-4387-9364-49794134943b resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22022025-04-17T00:32:05.984ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2580 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 79c46b53-bd14-4387-9364-49794134943b response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22032025-04-17T00:32:07.004ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d3e18f18-df80-461f-aa2f-f4e8766a400c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22042025-04-17T00:32:07.004ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d3e18f18-df80-461f-aa2f-f4e8766a400c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22052025-04-17T00:32:07.006ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2509 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d3e18f18-df80-461f-aa2f-f4e8766a400c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22062025-04-17T00:32:07.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
22072025-04-17T00:32:07.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22082025-04-17T00:32:07.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
22092025-04-17T00:32:07.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22102025-04-17T00:32:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22112025-04-17T00:32:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22122025-04-17T00:32:07.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22132025-04-17T00:32:07.964ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22142025-04-17T00:32:07.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22152025-04-17T00:32:07.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22162025-04-17T00:32:07.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22172025-04-17T00:32:07.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22182025-04-17T00:32:07.966ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.55145ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22192025-04-17T00:32:07.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22202025-04-17T00:32:07.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22212025-04-17T00:32:07.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22222025-04-17T00:32:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
22232025-04-17T00:32:07.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22242025-04-17T00:32:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22252025-04-17T00:32:07.979ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
22262025-04-17T00:32:07.979ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.887856ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
22272025-04-17T00:32:08.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 02c0726f-2acc-4cff-920c-e48c8b5faf36 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22282025-04-17T00:32:08.027ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 02c0726f-2acc-4cff-920c-e48c8b5faf36 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22292025-04-17T00:32:08.029ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1998 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 02c0726f-2acc-4cff-920c-e48c8b5faf36 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22302025-04-17T00:32:09.048ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 87cbaa76-c480-46a6-993d-a7c818e995c4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22312025-04-17T00:32:09.048ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 87cbaa76-c480-46a6-993d-a7c818e995c4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22322025-04-17T00:32:09.049ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2089 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 87cbaa76-c480-46a6-993d-a7c818e995c4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22332025-04-17T00:32:10.070ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6a88510-99a1-4f39-8004-10d030b7efa1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22342025-04-17T00:32:10.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6a88510-99a1-4f39-8004-10d030b7efa1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22352025-04-17T00:32:10.072ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2558 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6a88510-99a1-4f39-8004-10d030b7efa1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22362025-04-17T00:32:11.092ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 19e41f06-c69a-4dfd-af07-ee348287cccf roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22372025-04-17T00:32:11.093ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 19e41f06-c69a-4dfd-af07-ee348287cccf resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22382025-04-17T00:32:11.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 19e41f06-c69a-4dfd-af07-ee348287cccf response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22392025-04-17T00:32:11.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 8ff16996-b51b-4aa7-b06d-056d19dc4f4f response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
22402025-04-17T00:32:12.115ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8ac2bdd9-4b40-443f-bef5-b9945b44cb05 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22412025-04-17T00:32:12.115ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8ac2bdd9-4b40-443f-bef5-b9945b44cb05 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22422025-04-17T00:32:12.117ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1698 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8ac2bdd9-4b40-443f-bef5-b9945b44cb05 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22432025-04-17T00:32:13.137ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5dfe78af-8398-43e4-a6ac-4bc474166d8a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22442025-04-17T00:32:13.137ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5dfe78af-8398-43e4-a6ac-4bc474166d8a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22452025-04-17T00:32:13.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2166 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5dfe78af-8398-43e4-a6ac-4bc474166d8a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22462025-04-17T00:32:14.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = edce3e0d-d073-47a9-b55e-c357e682f7a4 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22472025-04-17T00:32:14.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = edce3e0d-d073-47a9-b55e-c357e682f7a4 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22482025-04-17T00:32:14.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2329 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = edce3e0d-d073-47a9-b55e-c357e682f7a4 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22492025-04-17T00:32:15.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f3731f6a-7de5-4724-96d8-a573e3257fa1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22502025-04-17T00:32:15.193ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f3731f6a-7de5-4724-96d8-a573e3257fa1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22512025-04-17T00:32:15.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3438 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f3731f6a-7de5-4724-96d8-a573e3257fa1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22522025-04-17T00:32:16.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9db805da-2b93-42ef-93cc-f6e2839e7a1e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22532025-04-17T00:32:16.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9db805da-2b93-42ef-93cc-f6e2839e7a1e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22542025-04-17T00:32:16.230ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3679 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9db805da-2b93-42ef-93cc-f6e2839e7a1e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22552025-04-17T00:32:17.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a05be6bf-12d0-4f85-826c-751faa2db8fc roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22562025-04-17T00:32:17.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a05be6bf-12d0-4f85-826c-751faa2db8fc resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22572025-04-17T00:32:17.263ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2102 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a05be6bf-12d0-4f85-826c-751faa2db8fc response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
22582025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
22592025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
22602025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
22612025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22622025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
22632025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22642025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
22652025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
22662025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22672025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
22682025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
22692025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
22702025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
22712025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22722025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
22732025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
22742025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22752025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22762025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
22772025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
22782025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22792025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
22802025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
22812025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
22822025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
22832025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22842025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
22852025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
22862025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
22872025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22882025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
22892025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
22902025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22912025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22922025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
22932025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
22942025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
22952025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
22962025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
22972025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
22982025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22992025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23002025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
23012025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
23022025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
23032025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23042025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
23052025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23062025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
23072025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23082025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
23092025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
23102025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
23112025-04-17T00:32:17.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
23122025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23132025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
23142025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
23152025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
23162025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
23172025-04-17T00:32:17.961ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
23182025-04-17T00:32:17.961ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-17 00:22:17.961556460 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
23192025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
23202025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
23212025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
23222025-04-17T00:32:17.961ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3
23232025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.641\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3"}
23242025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23252025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23262025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23272025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
23282025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23292025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23302025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
23312025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23322025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23332025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
23342025-04-17T00:32:17.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23352025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23362025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23372025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23382025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23392025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23402025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23412025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
23422025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23432025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
23442025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
23452025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23462025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
23472025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
23482025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23492025-04-17T00:32:17.962ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99145431-02b3-4b6e-aa5d-968f96332449.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1863, tv_nsec: 632836993 } })
23502025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23512025-04-17T00:32:17.962ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b3e44f3-f539-46b4-bb67-d442c7ebd3cf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4488d95c-7f86-4ca6-ab72-2e345a6a57bb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1863, tv_nsec: 632854773 } })
23522025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
23532025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.293873ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
23542025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
23552025-04-17T00:32:17.962ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.547919ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
23562025-04-17T00:32:17.967ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1863, tv_nsec: 638450854 } })
23572025-04-17T00:32:17.967ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1863, tv_nsec: 638586998 } })
23582025-04-17T00:32:17.968ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
23592025-04-17T00:32:17.968ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
23602025-04-17T00:32:17.974ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.951536ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
23612025-04-17T00:32:17.975ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23622025-04-17T00:32:17.975ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23632025-04-17T00:32:17.978ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23642025-04-17T00:32:17.978ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23652025-04-17T00:32:17.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23662025-04-17T00:32:17.978ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23672025-04-17T00:32:17.980ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
23682025-04-17T00:32:17.980ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23692025-04-17T00:32:17.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23702025-04-17T00:32:17.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23712025-04-17T00:32:17.981ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23722025-04-17T00:32:17.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23732025-04-17T00:32:17.984ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23742025-04-17T00:32:17.987ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23752025-04-17T00:32:17.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23762025-04-17T00:32:17.987ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23772025-04-17T00:32:17.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23782025-04-17T00:32:17.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23792025-04-17T00:32:17.989ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23802025-04-17T00:32:17.989ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23812025-04-17T00:32:17.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23822025-04-17T00:32:17.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23832025-04-17T00:32:17.995ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
23842025-04-17T00:32:17.995ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23852025-04-17T00:32:17.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23862025-04-17T00:32:17.996ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23872025-04-17T00:32:17.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23882025-04-17T00:32:17.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23892025-04-17T00:32:17.998ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-17 00:22:17.961556460 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
23902025-04-17T00:32:17.998ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 36.855598ms iteration = 12 status = {"expiration":"2025-04-17T00:22:17.961556460Z","pruned":[]}
23912025-04-17T00:32:17.999ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 37.625818ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
23922025-04-17T00:32:18.001ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23932025-04-17T00:32:18.004ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23942025-04-17T00:32:18.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23952025-04-17T00:32:18.004ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 43.138827ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
23962025-04-17T00:32:18.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
23972025-04-17T00:32:18.005ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
23982025-04-17T00:32:18.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
23992025-04-17T00:32:18.006ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24002025-04-17T00:32:18.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24012025-04-17T00:32:18.007ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24022025-04-17T00:32:18.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24032025-04-17T00:32:18.009ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24042025-04-17T00:32:18.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24052025-04-17T00:32:18.010ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
24062025-04-17T00:32:18.010ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24072025-04-17T00:32:18.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24082025-04-17T00:32:18.012ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24092025-04-17T00:32:18.012ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24102025-04-17T00:32:18.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24112025-04-17T00:32:18.014ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1863, tv_nsec: 684992336 } })
24122025-04-17T00:32:18.014ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
24132025-04-17T00:32:18.015ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24142025-04-17T00:32:18.016ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24152025-04-17T00:32:18.017ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24162025-04-17T00:32:18.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24172025-04-17T00:32:18.020ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24182025-04-17T00:32:18.020ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24192025-04-17T00:32:18.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24202025-04-17T00:32:18.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
24212025-04-17T00:32:18.021ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
24222025-04-17T00:32:18.021ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
24232025-04-17T00:32:18.021ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
24242025-04-17T00:32:18.021ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24252025-04-17T00:32:18.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24262025-04-17T00:32:18.022ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24272025-04-17T00:32:18.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24282025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
24292025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
24302025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
24312025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24322025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24332025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24342025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24352025-04-17T00:32:18.024ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24362025-04-17T00:32:18.026ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24372025-04-17T00:32:18.029ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
24382025-04-17T00:32:18.029ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24392025-04-17T00:32:18.029ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24402025-04-17T00:32:18.029ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24412025-04-17T00:32:18.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24422025-04-17T00:32:18.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24432025-04-17T00:32:18.031ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 70.207928ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
24442025-04-17T00:32:18.032ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24452025-04-17T00:32:18.033ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24462025-04-17T00:32:18.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
24472025-04-17T00:32:18.037ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24482025-04-17T00:32:18.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24492025-04-17T00:32:18.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24502025-04-17T00:32:18.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24512025-04-17T00:32:18.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
24522025-04-17T00:32:18.045ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24532025-04-17T00:32:18.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24542025-04-17T00:32:18.054ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 92.899051ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
24552025-04-17T00:32:18.055ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24562025-04-17T00:32:18.055ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
24572025-04-17T00:32:18.055ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24582025-04-17T00:32:18.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24592025-04-17T00:32:18.057ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
24602025-04-17T00:32:18.058ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
24612025-04-17T00:32:18.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24622025-04-17T00:32:18.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24632025-04-17T00:32:18.058ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
24642025-04-17T00:32:18.058ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 0 version = 2
24652025-04-17T00:32:18.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24662025-04-17T00:32:18.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24672025-04-17T00:32:18.060ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24682025-04-17T00:32:18.060ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
24692025-04-17T00:32:18.060ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24702025-04-17T00:32:18.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24712025-04-17T00:32:18.076ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24722025-04-17T00:32:18.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24732025-04-17T00:32:18.078ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24742025-04-17T00:32:18.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24752025-04-17T00:32:18.085ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
24762025-04-17T00:32:18.085ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
24772025-04-17T00:32:18.085ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 123.980533ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
24782025-04-17T00:32:18.086ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
24792025-04-17T00:32:18.086ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
24802025-04-17T00:32:18.086ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 124.615369ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
24812025-04-17T00:32:18.088ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24822025-04-17T00:32:18.090ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 129.05651ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24832025-04-17T00:32:18.090ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24842025-04-17T00:32:18.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24852025-04-17T00:32:18.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24862025-04-17T00:32:18.091ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24872025-04-17T00:32:18.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24882025-04-17T00:32:18.095ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 134.20108ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24892025-04-17T00:32:18.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24902025-04-17T00:32:18.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
24912025-04-17T00:32:18.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24922025-04-17T00:32:18.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24932025-04-17T00:32:18.107ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 146.027238ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24942025-04-17T00:32:18.118ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
24952025-04-17T00:32:18.119ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24962025-04-17T00:32:18.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
24972025-04-17T00:32:18.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24982025-04-17T00:32:18.139ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
24992025-04-17T00:32:18.139ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25002025-04-17T00:32:18.140ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5633fa03-1c00-484d-bd71-77f96dcf6225", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
25012025-04-17T00:32:18.140ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25022025-04-17T00:32:18.141ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25032025-04-17T00:32:18.171ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
25042025-04-17T00:32:18.181ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25052025-04-17T00:32:18.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25062025-04-17T00:32:18.191ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
25072025-04-17T00:32:18.198ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
25082025-04-17T00:32:18.198ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25092025-04-17T00:32:18.199ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "be0f87f9-a0d6-4af3-8a82-d79a84226788", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
25102025-04-17T00:32:18.199ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25112025-04-17T00:32:18.201ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25122025-04-17T00:32:18.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4b9001e1-212b-45f8-bd4c-7917bdcb6d37 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
25132025-04-17T00:32:18.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4b9001e1-212b-45f8-bd4c-7917bdcb6d37 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
25142025-04-17T00:32:18.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25152025-04-17T00:32:18.306ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9/state
25162025-04-17T00:32:18.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25172025-04-17T00:32:18.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25182025-04-17T00:32:18.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25192025-04-17T00:32:18.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25202025-04-17T00:32:18.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25212025-04-17T00:32:18.310ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25222025-04-17T00:32:18.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25232025-04-17T00:32:18.333ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25242025-04-17T00:32:18.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25252025-04-17T00:32:18.345ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25262025-04-17T00:32:18.345ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
25272025-04-17T00:32:18.345ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 383.856518ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
25282025-04-17T00:32:18.359ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
25292025-04-17T00:32:18.359ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25302025-04-17T00:32:18.359ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f354e85-e893-45ea-95c1-945426db0a17", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
25312025-04-17T00:32:18.359ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25322025-04-17T00:32:18.362ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
25332025-04-17T00:32:18.377ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
25342025-04-17T00:32:18.377ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25352025-04-17T00:32:18.378ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "47a5d335-9040-40f8-9c22-51e57e3a84f7", "content-length": "525", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
25362025-04-17T00:32:18.378ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25372025-04-17T00:32:18.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25382025-04-17T00:32:18.385ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25392025-04-17T00:32:18.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25402025-04-17T00:32:18.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 109593 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4b9001e1-212b-45f8-bd4c-7917bdcb6d37 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
25412025-04-17T00:32:18.397ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 1 version = 2
25422025-04-17T00:32:18.397ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
25432025-04-17T00:32:18.397ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
25442025-04-17T00:32:18.397ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-17 00:21:19.348689 UTC dns_group = external generation = 2
25452025-04-17T00:32:18.397ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 436.486624ms iteration = 12 status = {"generation":2}
25462025-04-17T00:32:18.397ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 436.345301ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
25472025-04-17T00:32:18.399ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
25482025-04-17T00:32:18.399ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
25492025-04-17T00:32:18.399ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-17 00:21:18.231041 UTC dns_group = internal generation = 1
25502025-04-17T00:32:18.399ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 437.676714ms iteration = 12 status = {"generation":1}
25512025-04-17T00:32:18.400ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25522025-04-17T00:32:18.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
25532025-04-17T00:32:18.405ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25542025-04-17T00:32:18.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25552025-04-17T00:32:18.410ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25562025-04-17T00:32:18.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25572025-04-17T00:32:18.416ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 455.361759ms iteration = 23 status = 0
25582025-04-17T00:32:18.422ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 461.035173ms iteration = 23 status = null
25592025-04-17T00:32:18.425ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 464.428918ms iteration = 23 status = {}
25602025-04-17T00:32:18.428ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5c0bb522-3907-4f66-844b-34273cf292c1: {10.151.1.176: {83dd7861-7433-4cff-8f6c-8052657f8c96}, 10.151.1.175: {83dd7861-7433-4cff-8f6c-8052657f8c96}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
25612025-04-17T00:32:18.428ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
25622025-04-17T00:32:18.429ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d75d4049-d361-44c3-9a63-80f90067fc4a", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
25632025-04-17T00:32:18.429ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25642025-04-17T00:32:18.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25652025-04-17T00:32:18.431ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25662025-04-17T00:32:18.432ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25672025-04-17T00:32:18.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25682025-04-17T00:32:18.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25692025-04-17T00:32:18.434ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25702025-04-17T00:32:18.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25712025-04-17T00:32:18.435ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
25722025-04-17T00:32:18.436ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d5040b7f-a269-425d-987c-f5740544adba", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25732025-04-17T00:32:18.436ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25742025-04-17T00:32:18.438ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "9e68451d-dc2a-4f26-a009-9565e2f3204f", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25752025-04-17T00:32:18.438ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25762025-04-17T00:32:18.438ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
25772025-04-17T00:32:18.438ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 477.387395ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
25782025-04-17T00:32:18.441ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25792025-04-17T00:32:18.441ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 479.932009ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
25802025-04-17T00:32:18.441ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25812025-04-17T00:32:18.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25822025-04-17T00:32:18.443ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25832025-04-17T00:32:18.445ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25842025-04-17T00:32:18.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25852025-04-17T00:32:18.446ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25862025-04-17T00:32:18.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25872025-04-17T00:32:18.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25882025-04-17T00:32:18.448ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25892025-04-17T00:32:18.448ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25902025-04-17T00:32:18.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25912025-04-17T00:32:18.448ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 487.505689ms iteration = 26 status = {"physical_disks_added":0}
25922025-04-17T00:32:18.450ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25932025-04-17T00:32:18.450ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25942025-04-17T00:32:18.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25952025-04-17T00:32:18.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25962025-04-17T00:32:18.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25972025-04-17T00:32:18.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25982025-04-17T00:32:18.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25992025-04-17T00:32:18.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26002025-04-17T00:32:18.456ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26012025-04-17T00:32:18.456ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26022025-04-17T00:32:18.456ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
26032025-04-17T00:32:18.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26042025-04-17T00:32:18.458ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26052025-04-17T00:32:18.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26062025-04-17T00:32:18.460ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 498.964738ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
26072025-04-17T00:32:18.460ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26082025-04-17T00:32:18.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26092025-04-17T00:32:18.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26102025-04-17T00:32:18.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26112025-04-17T00:32:18.464ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26122025-04-17T00:32:18.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26132025-04-17T00:32:18.465ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26142025-04-17T00:32:18.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26152025-04-17T00:32:18.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26162025-04-17T00:32:18.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26172025-04-17T00:32:18.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26182025-04-17T00:32:18.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: fbd6d1cd-e3a0-472d-85fc-87b80458cedc, lookup_type: ById(fbd6d1cd-e3a0-472d-85fc-87b80458cedc) }, key: 10d2410d-7f4e-439c-b44c-be0210ddf7a2, lookup_type: ById(10d2410d-7f4e-439c-b44c-be0210ddf7a2) } result = Ok(())
26192025-04-17T00:32:18.509ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26202025-04-17T00:32:18.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26212025-04-17T00:32:18.509ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-17T00:21:15.858908Z, time_modified: 2025-04-17T00:21:15.858908Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
26222025-04-17T00:32:18.509ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 548.185518ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]}},"default_endpoint":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
26232025-04-17T00:32:18.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26242025-04-17T00:32:18.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26252025-04-17T00:32:18.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26262025-04-17T00:32:18.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26272025-04-17T00:32:18.521ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
26282025-04-17T00:32:18.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26292025-04-17T00:32:18.521ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "c65318f6-61f1-4162-8372-b08153804c61", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26302025-04-17T00:32:18.521ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26312025-04-17T00:32:18.523ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7efc96d7-74ad-4161-bda8-153b332a401d", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26322025-04-17T00:32:18.523ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26332025-04-17T00:32:18.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: fbd6d1cd-e3a0-472d-85fc-87b80458cedc, lookup_type: ById(fbd6d1cd-e3a0-472d-85fc-87b80458cedc) }, key: 10d2410d-7f4e-439c-b44c-be0210ddf7a2, lookup_type: ById(10d2410d-7f4e-439c-b44c-be0210ddf7a2) } result = Ok(())
26342025-04-17T00:32:18.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26352025-04-17T00:32:18.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26362025-04-17T00:32:18.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a59a856e-1c74-42d1-84d1-8c6e7a79c60e", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26372025-04-17T00:32:18.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26382025-04-17T00:32:18.528ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
26392025-04-17T00:32:18.528ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 566.853548ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
26402025-04-17T00:32:18.528ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26412025-04-17T00:32:18.528ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 566.684633ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
26422025-04-17T00:32:18.529ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26432025-04-17T00:32:18.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26442025-04-17T00:32:18.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26452025-04-17T00:32:18.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26462025-04-17T00:32:18.534ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26472025-04-17T00:32:18.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26482025-04-17T00:32:18.536ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26492025-04-17T00:32:18.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26502025-04-17T00:32:18.538ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26512025-04-17T00:32:18.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26522025-04-17T00:32:18.539ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26532025-04-17T00:32:18.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26542025-04-17T00:32:18.541ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26552025-04-17T00:32:18.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26562025-04-17T00:32:18.558ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26572025-04-17T00:32:18.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26582025-04-17T00:32:18.577ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26592025-04-17T00:32:18.577ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "02a2ac62-a141-47b0-a6c8-78243bbb53e2", "content-length": "811", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
26602025-04-17T00:32:18.577ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(11334347) }} sled = buskin
26612025-04-17T00:32:18.577ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
26622025-04-17T00:32:18.577ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
26632025-04-17T00:32:18.578ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 617.224447ms iteration = 23 status = {}
26642025-04-17T00:32:18.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26652025-04-17T00:32:18.579ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
26662025-04-17T00:32:18.579ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26672025-04-17T00:32:18.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26682025-04-17T00:32:18.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: fbd6d1cd-e3a0-472d-85fc-87b80458cedc, lookup_type: ById(fbd6d1cd-e3a0-472d-85fc-87b80458cedc) }, key: 9c831c23-6953-47b0-809d-56a7d0fcc946, lookup_type: ById(9c831c23-6953-47b0-809d-56a7d0fcc946) } result = Ok(())
26692025-04-17T00:32:18.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26702025-04-17T00:32:18.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26712025-04-17T00:32:18.584ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26722025-04-17T00:32:18.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26732025-04-17T00:32:18.587ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26742025-04-17T00:32:18.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26752025-04-17T00:32:18.588ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26762025-04-17T00:32:18.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26772025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
26782025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
26792025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
26802025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26812025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26822025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26832025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26842025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26852025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26862025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26872025-04-17T00:32:18.589ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
26882025-04-17T00:32:18.589ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26892025-04-17T00:32:18.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26902025-04-17T00:32:18.591ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26912025-04-17T00:32:18.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26922025-04-17T00:32:18.592ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
26932025-04-17T00:32:18.592ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26942025-04-17T00:32:18.592ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26952025-04-17T00:32:18.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26962025-04-17T00:32:18.594ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26972025-04-17T00:32:18.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26982025-04-17T00:32:18.595ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26992025-04-17T00:32:18.599ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: fbd6d1cd-e3a0-472d-85fc-87b80458cedc, lookup_type: ById(fbd6d1cd-e3a0-472d-85fc-87b80458cedc) }, key: 9c831c23-6953-47b0-809d-56a7d0fcc946, lookup_type: ByName("default") } result = Ok(())
27002025-04-17T00:32:18.599ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27012025-04-17T00:32:18.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27022025-04-17T00:32:18.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27032025-04-17T00:32:18.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27042025-04-17T00:32:18.603ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27052025-04-17T00:32:18.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27062025-04-17T00:32:18.605ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27072025-04-17T00:32:18.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27082025-04-17T00:32:18.608ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27092025-04-17T00:32:18.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27102025-04-17T00:32:18.609ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27112025-04-17T00:32:18.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27122025-04-17T00:32:18.611ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27132025-04-17T00:32:18.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27142025-04-17T00:32:18.612ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27152025-04-17T00:32:18.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: fbd6d1cd-e3a0-472d-85fc-87b80458cedc, lookup_type: ById(fbd6d1cd-e3a0-472d-85fc-87b80458cedc) }, key: 83dd7861-7433-4cff-8f6c-8052657f8c96, lookup_type: ByName("default") } result = Ok(())
27162025-04-17T00:32:18.616ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
27172025-04-17T00:32:18.616ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27182025-04-17T00:32:18.692ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cade5ed8-ec66-4962-a45a-eaa43e870bd2", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
27192025-04-17T00:32:18.692ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
27202025-04-17T00:32:18.692ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27212025-04-17T00:32:18.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "88242260-ca46-4d81-bb34-296db9cc7cff", "content-length": "682", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
27222025-04-17T00:32:18.693ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27232025-04-17T00:32:18.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27242025-04-17T00:32:18.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27252025-04-17T00:32:18.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27262025-04-17T00:32:18.698ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ce10dfc4-8993-44d2-9c30-0739f5e97548", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
27272025-04-17T00:32:18.700ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 739.423534ms iteration = 23 status = {}
27282025-04-17T00:32:18.701ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27292025-04-17T00:32:18.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27302025-04-17T00:32:18.703ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27312025-04-17T00:32:18.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27322025-04-17T00:32:18.710ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 749.746154ms iteration = 23 status = {}
27332025-04-17T00:32:18.881ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
27342025-04-17T00:32:18.881ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::a]:12221
27352025-04-17T00:32:18.913ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
27362025-04-17T00:32:18.913ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:35610
27372025-04-17T00:32:18.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35610 req_id = 08fefde9-a538-406b-a972-c4ad54eae7a9 roles = RoleSet { roles: {} } uri = /metrics/producers
27382025-04-17T00:32:18.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35610 req_id = 08fefde9-a538-406b-a972-c4ad54eae7a9 resource = Database result = Ok(()) uri = /metrics/producers
27392025-04-17T00:32:18.947ZINFOnexus: assigned collector to new producer collector_id = c581c159-53e3-4159-ba13-1f9ca3dc97cc file = nexus/src/app/oximeter.rs:76 producer_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
27402025-04-17T00:32:18.947ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 34436 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35610 req_id = 08fefde9-a538-406b-a972-c4ad54eae7a9 response_code = 201 uri = /metrics/producers
27412025-04-17T00:32:18.947ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "08fefde9-a538-406b-a972-c4ad54eae7a9", "content-length": "41", "date": "Thu, 17 Apr 2025 00:32:18 GMT"} })
27422025-04-17T00:32:18.949ZDEBGnexus (producer-registration-task): registered with nexus successfully
27432025-04-17T00:32:18.949ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
27442025-04-17T00:32:19.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = dbac9c7b-4ab4-4a21-910a-1832c9dc3f81 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27452025-04-17T00:32:19.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = dbac9c7b-4ab4-4a21-910a-1832c9dc3f81 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27462025-04-17T00:32:19.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = dbac9c7b-4ab4-4a21-910a-1832c9dc3f81 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27472025-04-17T00:32:20.446ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 69eff06d-accf-4f4e-b3e5-e6b0a4508213 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27482025-04-17T00:32:20.446ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 69eff06d-accf-4f4e-b3e5-e6b0a4508213 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27492025-04-17T00:32:20.448ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2521 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 69eff06d-accf-4f4e-b3e5-e6b0a4508213 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27502025-04-17T00:32:21.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 72c68e18-f461-4a56-a282-43035a8b1740 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27512025-04-17T00:32:21.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 72c68e18-f461-4a56-a282-43035a8b1740 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27522025-04-17T00:32:21.470ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2572 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 72c68e18-f461-4a56-a282-43035a8b1740 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27532025-04-17T00:32:21.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 467 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 9ff744a4-eaae-4dc3-be7a-b25b0e06e431 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
27542025-04-17T00:32:22.124ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ac29bee-3056-4171-924d-dea6b498b342", "content-length": "131", "date": "Thu, 17 Apr 2025 00:32:22 GMT"} })
27552025-04-17T00:32:22.129ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 sled_id = ca62f5ab-0935-4588-a5c0-db1db0e32680 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-17T00:32:13.776226527Z }, migration_in: None, migration_out: None } vmm_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9
27562025-04-17T00:32:22.129ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2168 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 sled_id = ca62f5ab-0935-4588-a5c0-db1db0e32680 vmm_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-17T00:32:13.776226527Z }
27572025-04-17T00:32:22.129ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 roles = RoleSet { roles: {} } sled_id = ca62f5ab-0935-4588-a5c0-db1db0e32680 vmm_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9
27582025-04-17T00:32:22.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 resource = Database result = Ok(()) sled_id = ca62f5ab-0935-4588-a5c0-db1db0e32680 vmm_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9
27592025-04-17T00:32:22.134ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca43c9f0-54e9-427d-8a62-377f18d6bdc6 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-04-17T00:32:22.124251795Z }
27602025-04-17T00:32:22.134ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca43c9f0-54e9-427d-8a62-377f18d6bdc6 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
27612025-04-17T00:32:22.135ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca43c9f0-54e9-427d-8a62-377f18d6bdc6 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
27622025-04-17T00:32:22.182ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
27632025-04-17T00:32:22.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 59471 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ca43c9f0-54e9-427d-8a62-377f18d6bdc6 response_code = 204 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
27642025-04-17T00:32:22.184ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 4.223182861s iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"starting":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
27652025-04-17T00:32:22.213ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b692d261-9159-4ffa-864d-57b944ea31e0 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-04-17T00:32:22.213577765Z }
27662025-04-17T00:32:22.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b692d261-9159-4ffa-864d-57b944ea31e0 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
27672025-04-17T00:32:22.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b692d261-9159-4ffa-864d-57b944ea31e0 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
27682025-04-17T00:32:22.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 40520 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b692d261-9159-4ffa-864d-57b944ea31e0 response_code = 204 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
27692025-04-17T00:32:22.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 74a8d59e-89ad-4e4b-880e-61b622c0036c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27702025-04-17T00:32:22.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 74a8d59e-89ad-4e4b-880e-61b622c0036c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27712025-04-17T00:32:22.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2062 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 74a8d59e-89ad-4e4b-880e-61b622c0036c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27722025-04-17T00:32:23.510ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d3d608cc-b2be-4821-bf75-608252112c69 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27732025-04-17T00:32:23.511ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d3d608cc-b2be-4821-bf75-608252112c69 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27742025-04-17T00:32:23.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2507 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d3d608cc-b2be-4821-bf75-608252112c69 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27752025-04-17T00:32:24.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 19ed5b6d-7c8c-4318-b21d-8f49d91574a6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27762025-04-17T00:32:24.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 19ed5b6d-7c8c-4318-b21d-8f49d91574a6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27772025-04-17T00:32:24.536ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2545 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 19ed5b6d-7c8c-4318-b21d-8f49d91574a6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27782025-04-17T00:32:25.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b33ee624-06c7-41e9-92bd-87289bd13c82 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27792025-04-17T00:32:25.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b33ee624-06c7-41e9-92bd-87289bd13c82 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27802025-04-17T00:32:25.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3083 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b33ee624-06c7-41e9-92bd-87289bd13c82 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27812025-04-17T00:32:26.583ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eda3a425-7da3-4569-91b5-d68d6fd5d540 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27822025-04-17T00:32:26.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eda3a425-7da3-4569-91b5-d68d6fd5d540 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27832025-04-17T00:32:26.585ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2638 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = eda3a425-7da3-4569-91b5-d68d6fd5d540 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27842025-04-17T00:32:27.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8bcfb0da-5bee-4192-a3bc-6ffc2d7162f2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27852025-04-17T00:32:27.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8bcfb0da-5bee-4192-a3bc-6ffc2d7162f2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27862025-04-17T00:32:27.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8bcfb0da-5bee-4192-a3bc-6ffc2d7162f2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
27872025-04-17T00:32:27.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
27882025-04-17T00:32:27.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
27892025-04-17T00:32:27.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27902025-04-17T00:32:27.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27912025-04-17T00:32:27.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27922025-04-17T00:32:27.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27932025-04-17T00:32:27.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27942025-04-17T00:32:27.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27952025-04-17T00:32:27.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27962025-04-17T00:32:27.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27972025-04-17T00:32:27.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27982025-04-17T00:32:27.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27992025-04-17T00:32:27.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28002025-04-17T00:32:27.976ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
28012025-04-17T00:32:27.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28022025-04-17T00:32:27.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28032025-04-17T00:32:27.988ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
28042025-04-17T00:32:27.988ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.201605ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
28052025-04-17T00:32:27.998ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28062025-04-17T00:32:27.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28072025-04-17T00:32:28.027ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.443733ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28082025-04-17T00:32:28.628ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 649d87f1-a0b2-4a75-aa20-251160eb518a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28092025-04-17T00:32:28.628ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 649d87f1-a0b2-4a75-aa20-251160eb518a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28102025-04-17T00:32:28.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2039 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 649d87f1-a0b2-4a75-aa20-251160eb518a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28112025-04-17T00:32:29.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 08b93a52-4e9c-450b-a4aa-5a6cc3c9f7f0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28122025-04-17T00:32:29.649ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 08b93a52-4e9c-450b-a4aa-5a6cc3c9f7f0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28132025-04-17T00:32:29.651ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 08b93a52-4e9c-450b-a4aa-5a6cc3c9f7f0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28142025-04-17T00:32:30.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4ca9e372-845e-412d-814c-3acaa31bc14a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28152025-04-17T00:32:30.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4ca9e372-845e-412d-814c-3acaa31bc14a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28162025-04-17T00:32:30.672ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2455 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4ca9e372-845e-412d-814c-3acaa31bc14a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28172025-04-17T00:32:31.534ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f92ff18c-c955-492f-9fae-c1b335f35153 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-04-17T00:32:31.534539053Z }
28182025-04-17T00:32:31.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f92ff18c-c955-492f-9fae-c1b335f35153 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28192025-04-17T00:32:31.535ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f92ff18c-c955-492f-9fae-c1b335f35153 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28202025-04-17T00:32:31.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 12748 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f92ff18c-c955-492f-9fae-c1b335f35153 response_code = 204 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28212025-04-17T00:32:31.638ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:42490
28222025-04-17T00:32:31.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 12c6664f-ccde-4cf8-b2a6-e82bf2d5d983 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
28232025-04-17T00:32:31.638ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 12c6664f-ccde-4cf8-b2a6-e82bf2d5d983 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
28242025-04-17T00:32:31.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2824 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 12c6664f-ccde-4cf8-b2a6-e82bf2d5d983 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
28252025-04-17T00:32:31.641ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 9b110e0e-894c-4a4f-ad3f-ce2ba3eec2f0 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
28262025-04-17T00:32:31.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 9b110e0e-894c-4a4f-ad3f-ce2ba3eec2f0 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
28272025-04-17T00:32:31.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2484 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 9b110e0e-894c-4a4f-ad3f-ce2ba3eec2f0 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
28282025-04-17T00:32:31.657ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 560 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = ad0e6cf1-a975-404c-9b7e-1227342ddb5f response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
28292025-04-17T00:32:31.693ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5bc60137-38f4-44b4-b72b-f11fe4b6f8d1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28302025-04-17T00:32:31.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5bc60137-38f4-44b4-b72b-f11fe4b6f8d1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28312025-04-17T00:32:31.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1727 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5bc60137-38f4-44b4-b72b-f11fe4b6f8d1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28322025-04-17T00:32:32.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 264d20c0-4899-4418-baf4-c1005ca5c81f roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28332025-04-17T00:32:32.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 264d20c0-4899-4418-baf4-c1005ca5c81f resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28342025-04-17T00:32:32.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2833 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 264d20c0-4899-4418-baf4-c1005ca5c81f response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28352025-04-17T00:32:33.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3c069c1b-c493-4120-9009-306cca4666c3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28362025-04-17T00:32:33.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3c069c1b-c493-4120-9009-306cca4666c3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28372025-04-17T00:32:33.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2200 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 3c069c1b-c493-4120-9009-306cca4666c3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
28382025-04-17T00:32:34.041ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-04-17T00:32:31.547927060Z }
28392025-04-17T00:32:34.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28402025-04-17T00:32:34.042ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28412025-04-17T00:32:34.098ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_needs_update = true
28422025-04-17T00:32:34.098ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28432025-04-17T00:32:34.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28442025-04-17T00:32:34.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28452025-04-17T00:32:34.104ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28462025-04-17T00:32:34.106ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28472025-04-17T00:32:34.107ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28482025-04-17T00:32:34.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28492025-04-17T00:32:34.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28502025-04-17T00:32:34.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28512025-04-17T00:32:34.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28522025-04-17T00:32:34.120ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28532025-04-17T00:32:34.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Database result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28542025-04-17T00:32:34.123ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28552025-04-17T00:32:34.127ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: 3b862e19-829e-4073-8318-fa518ae6d138, lookup_type: ById(3b862e19-829e-4073-8318-fa518ae6d138) } result = Ok(()) uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28562025-04-17T00:32:34.138ZINFOnexus (dropshot_internal): starting update saga for 3b862e19-829e-4073-8318-fa518ae6d138 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1631 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-04-17T00:32:31.547927060Z }
28572025-04-17T00:32:34.138ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 96954 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 response_code = 204 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28582025-04-17T00:32:34.138ZDEBGnexus (dropshot_internal): preparing instance-update saga for 3b862e19-829e-4073-8318-fa518ae6d138... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e46843fe-59b5-438f-b6cf-385c65cdf583 uri = /vmms/0eb88743-e5c4-471f-8837-aefa3728c3a9
28592025-04-17T00:32:34.138ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update
28602025-04-17T00:32:34.138ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"3b862e19-829e-4073-8318-fa518ae6d138","lookup_type":{"ById":"3b862e19-829e-4073-8318-fa518ae6d138"},"parent":{"key":"7ca11a5c-53f9-4c19-846c-c1c64ef8797a","lookup_type":{"ById":"7ca11a5c-53f9-4c19-846c-c1c64ef8797a"},"parent":{"key":"4614ac20-ae32-44af-af3c-2b45ace5dda2","lookup_type":{"ById":"4614ac20-ae32-44af-af3c-2b45ace5dda2"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28612025-04-17T00:32:34.138ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update
28622025-04-17T00:32:34.191ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update
28632025-04-17T00:32:34.191ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28642025-04-17T00:32:34.191ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28652025-04-17T00:32:34.191ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
28662025-04-17T00:32:34.204ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28672025-04-17T00:32:34.204ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
28682025-04-17T00:32:34.210ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28692025-04-17T00:32:34.210ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
28702025-04-17T00:32:34.219ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28712025-04-17T00:32:34.219ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("57b98b0c-6c46-4a3c-b0f7-3894d172878f")) node_id = 0 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
28722025-04-17T00:32:34.249ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28732025-04-17T00:32:34.249ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
28742025-04-17T00:32:34.280ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update
28752025-04-17T00:32:34.284ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28762025-04-17T00:32:34.285ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28772025-04-17T00:32:34.290ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28782025-04-17T00:32:34.290ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28792025-04-17T00:32:34.292ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28802025-04-17T00:32:34.293ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28812025-04-17T00:32:34.299ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28822025-04-17T00:32:34.300ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28832025-04-17T00:32:34.302ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28842025-04-17T00:32:34.302ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28852025-04-17T00:32:34.304ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28862025-04-17T00:32:34.304ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28872025-04-17T00:32:34.305ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28882025-04-17T00:32:34.311ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: 3b862e19-829e-4073-8318-fa518ae6d138, lookup_type: ById(3b862e19-829e-4073-8318-fa518ae6d138) } result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28892025-04-17T00:32:34.311ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance updater_id = 57b98b0c-6c46-4a3c-b0f7-3894d172878f
28902025-04-17T00:32:34.311ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28912025-04-17T00:32:34.311ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28922025-04-17T00:32:34.329ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance updater_id = 57b98b0c-6c46-4a3c-b0f7-3894d172878f
28932025-04-17T00:32:34.329ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28942025-04-17T00:32:34.329ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance
28952025-04-17T00:32:34.334ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 locked_gen = Generation(Generation(2)) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = LockInstance updater_id = 57b98b0c-6c46-4a3c-b0f7-3894d172878f
28962025-04-17T00:32:34.334ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28972025-04-17T00:32:34.334ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("57b98b0c-6c46-4a3c-b0f7-3894d172878f")}) node_id = 1 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
28982025-04-17T00:32:34.339ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
28992025-04-17T00:32:34.339ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
29002025-04-17T00:32:34.371ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29012025-04-17T00:32:34.371ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29022025-04-17T00:32:34.372ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29032025-04-17T00:32:34.372ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29042025-04-17T00:32:34.374ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29052025-04-17T00:32:34.374ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29062025-04-17T00:32:34.375ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29072025-04-17T00:32:34.379ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: 3b862e19-829e-4073-8318-fa518ae6d138, lookup_type: ById(3b862e19-829e-4073-8318-fa518ae6d138) } result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29082025-04-17T00:32:34.379ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29092025-04-17T00:32:34.379ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
29102025-04-17T00:32:34.392ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 0eb88743-e5c4-471f-8837-aefa3728c3a9, time_created: 2025-04-17T00:32:13.110597Z, time_deleted: None, instance_id: 3b862e19-829e-4073-8318-fa518ae6d138, sled_id: ca62f5ab-0935-4588-a5c0-db1db0e32680, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-04-17T00:32:31.547927Z, gen: Generation(Generation(7)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-04-17T00:32:10.981678Z, gen: Generation(Generation(3)), propolis_id: Some(0eb88743-e5c4-471f-8837-aefa3728c3a9), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(0eb88743-e5c4-471f-8837-aefa3728c3a9 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-04-17T00:32:34.392699683Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
29112025-04-17T00:32:34.397ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29122025-04-17T00:32:34.400ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"3b862e19-829e-4073-8318-fa518ae6d138","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"0eb88743-e5c4-471f-8837-aefa3728c3a9"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"3b862e19-829e-4073-8318-fa518ae6d138","lookup_type":{"ById":"3b862e19-829e-4073-8318-fa518ae6d138"},"parent":{"key":"7ca11a5c-53f9-4c19-846c-c1c64ef8797a","lookup_type":{"ById":"7ca11a5c-53f9-4c19-846c-c1c64ef8797a"},"parent":{"key":"4614ac20-ae32-44af-af3c-2b45ace5dda2","lookup_type":{"ById":"4614ac20-ae32-44af-af3c-2b45ace5dda2"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"57b98b0c-6c46-4a3c-b0f7-3894d172878f"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"7ca11a5c-53f9-4c19-846c-c1c64ef8797a","ram_diff":1073741824},"destroy_active_vmm":"0eb88743-e5c4-471f-8837-aefa3728c3a9","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-04-17T00:32:34.392699683Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29132025-04-17T00:32:34.400ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29142025-04-17T00:32:34.424ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29152025-04-17T00:32:34.424ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29162025-04-17T00:32:34.424ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29172025-04-17T00:32:34.424ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29182025-04-17T00:32:34.434ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29192025-04-17T00:32:34.434ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29202025-04-17T00:32:34.444ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29212025-04-17T00:32:34.444ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29222025-04-17T00:32:34.452ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29232025-04-17T00:32:34.452ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("47891844-9140-4f6c-809a-49c4294f8ff9")) node_id = 0 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29242025-04-17T00:32:34.457ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29252025-04-17T00:32:34.457ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29262025-04-17T00:32:34.466ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 parent_lock = UpdaterLock { updater_id: 57b98b0c-6c46-4a3c-b0f7-3894d172878f, locked_gen: Generation(Generation(2)) } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29272025-04-17T00:32:34.466ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = BecomeUpdater
29282025-04-17T00:32:34.466ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = BecomeUpdater
29292025-04-17T00:32:34.485ZDEBGnexus (SagaExecutor): inherited lock from 57b98b0c-6c46-4a3c-b0f7-3894d172878f to 47891844-9140-4f6c-809a-49c4294f8ff9 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 57b98b0c-6c46-4a3c-b0f7-3894d172878f saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = BecomeUpdater updater_id = 47891844-9140-4f6c-809a-49c4294f8ff9
29302025-04-17T00:32:34.485ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29312025-04-17T00:32:34.487ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29322025-04-17T00:32:34.487ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("47891844-9140-4f6c-809a-49c4294f8ff9")}) node_id = 1 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29332025-04-17T00:32:34.491ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29342025-04-17T00:32:34.491ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29352025-04-17T00:32:34.503ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29362025-04-17T00:32:34.503ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29372025-04-17T00:32:34.506ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29382025-04-17T00:32:34.506ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29392025-04-17T00:32:34.514ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29402025-04-17T00:32:34.514ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138
29412025-04-17T00:32:34.514ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UpdateNetworkConfig
29422025-04-17T00:32:34.514ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UpdateNetworkConfig
29432025-04-17T00:32:34.519ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UpdateNetworkConfig
29442025-04-17T00:32:34.519ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UpdateNetworkConfig
29452025-04-17T00:32:34.524ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
29462025-04-17T00:32:34.524ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UpdateNetworkConfig
29472025-04-17T00:32:34.524ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UpdateNetworkConfig
29482025-04-17T00:32:34.526ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
29492025-04-17T00:32:34.527ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
29502025-04-17T00:32:34.527ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
29512025-04-17T00:32:34.529ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
29522025-04-17T00:32:34.530ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
29532025-04-17T00:32:34.530ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
29542025-04-17T00:32:34.530ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
29552025-04-17T00:32:34.537ZDEBGnexus: notifying dendrite of updates instance_id = Some(3b862e19-829e-4073-8318-fa518ae6d138 (instance)) switch = switch0
29562025-04-17T00:32:34.538ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1880, tv_nsec: 209222067 } })
29572025-04-17T00:32:34.538ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1118
29582025-04-17T00:32:34.569ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
29592025-04-17T00:32:34.569ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29602025-04-17T00:32:34.569ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7bcd13c-b4ce-49c5-9d66-da6d1a8c7540", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:34 GMT"} })
29612025-04-17T00:32:34.569ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29622025-04-17T00:32:34.570ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29632025-04-17T00:32:34.600ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29642025-04-17T00:32:34.600ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29652025-04-17T00:32:34.605ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29662025-04-17T00:32:34.605ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29672025-04-17T00:32:34.609ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ReleaseVirtualProvisioning
29682025-04-17T00:32:34.609ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ReleaseVirtualProvisioning
29692025-04-17T00:32:34.662ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-04-17T00:32:34.612965Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, time_modified: Some(2025-04-17T00:32:34.612965Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, time_modified: Some(2025-04-17T00:32:34.612965Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29702025-04-17T00:32:34.662ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29712025-04-17T00:32:34.662ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29722025-04-17T00:32:34.666ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29732025-04-17T00:32:34.666ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29742025-04-17T00:32:34.672ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29752025-04-17T00:32:34.672ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UnassignOximeterProducer
29762025-04-17T00:32:34.672ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UnassignOximeterProducer
29772025-04-17T00:32:34.677ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = c581c159-53e3-4159-ba13-1f9ca3dc97cc producer_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29782025-04-17T00:32:34.677ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UnassignOximeterProducer
29792025-04-17T00:32:34.677ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = UnassignOximeterProducer
29802025-04-17T00:32:34.713ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29812025-04-17T00:32:34.713ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/3b862e19-829e-4073-8318-fa518ae6d138
29822025-04-17T00:32:34.736ZDEBGnexus (SagaExecutor): client response oximeter-collector = 3b862e19-829e-4073-8318-fa518ae6d138 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/3b862e19-829e-4073-8318-fa518ae6d138", status: 204, headers: {"x-request-id": "014a5fe3-07ca-4d52-ae87-64ef55427db3", "date": "Thu, 17 Apr 2025 00:32:34 GMT"} }) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29832025-04-17T00:32:34.736ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = c581c159-53e3-4159-ba13-1f9ca3dc97cc producer_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29842025-04-17T00:32:34.737ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29852025-04-17T00:32:34.737ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29862025-04-17T00:32:34.741ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29872025-04-17T00:32:34.741ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29882025-04-17T00:32:34.744ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 lock = UpdaterLock { updater_id: 47891844-9140-4f6c-809a-49c4294f8ff9, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-04-17T00:32:34.392699683Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29892025-04-17T00:32:34.744ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = CommitInstanceUpdates
29902025-04-17T00:32:34.744ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = CommitInstanceUpdates
29912025-04-17T00:32:34.761ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a85a7138-6cf5-461a-aceb-e645b531436f roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
29922025-04-17T00:32:34.761ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a85a7138-6cf5-461a-aceb-e645b531436f resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
29932025-04-17T00:32:34.763ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2054 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a85a7138-6cf5-461a-aceb-e645b531436f response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
29942025-04-17T00:32:34.784ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 new_runtime = InstanceRuntimeState { time_updated: 2025-04-17T00:32:34.392699683Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
29952025-04-17T00:32:34.784ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
29962025-04-17T00:32:34.784ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
29972025-04-17T00:32:34.784ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
29982025-04-17T00:32:34.784ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
29992025-04-17T00:32:34.784ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
30002025-04-17T00:32:34.784ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30012025-04-17T00:32:34.784ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30022025-04-17T00:32:34.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30032025-04-17T00:32:34.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30042025-04-17T00:32:34.787ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30052025-04-17T00:32:34.788ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30062025-04-17T00:32:34.788ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30072025-04-17T00:32:34.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30082025-04-17T00:32:34.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30092025-04-17T00:32:34.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30102025-04-17T00:32:34.790ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30112025-04-17T00:32:34.792ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30122025-04-17T00:32:34.792ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("3b862e19-829e-4073-8318-fa518ae6d138"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("0eb88743-e5c4-471f-8837-aefa3728c3a9")}) node_id = 7 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30132025-04-17T00:32:34.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30142025-04-17T00:32:34.792ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30152025-04-17T00:32:34.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30162025-04-17T00:32:34.795ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30172025-04-17T00:32:34.795ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30182025-04-17T00:32:34.797ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30192025-04-17T00:32:34.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30202025-04-17T00:32:34.799ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30212025-04-17T00:32:34.799ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30222025-04-17T00:32:34.802ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30232025-04-17T00:32:34.802ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30242025-04-17T00:32:34.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30252025-04-17T00:32:34.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30262025-04-17T00:32:34.806ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
30272025-04-17T00:32:34.806ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ReleaseSledResources
30282025-04-17T00:32:34.806ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ReleaseSledResources
30292025-04-17T00:32:34.828ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30302025-04-17T00:32:34.828ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30312025-04-17T00:32:34.828ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "15443d5d-dfa4-4626-b36c-0b819f0f7696", "content-length": "372", "date": "Thu, 17 Apr 2025 00:32:34 GMT"} })
30322025-04-17T00:32:34.828ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30332025-04-17T00:32:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30342025-04-17T00:32:34.830ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30352025-04-17T00:32:34.832ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30362025-04-17T00:32:34.832ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30372025-04-17T00:32:34.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30382025-04-17T00:32:34.834ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30392025-04-17T00:32:34.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30402025-04-17T00:32:34.835ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30412025-04-17T00:32:34.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30422025-04-17T00:32:34.845ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5c0bb522-3907-4f66-844b-34273cf292c1: {10.151.1.176: {83dd7861-7433-4cff-8f6c-8052657f8c96}, 10.151.1.175: {83dd7861-7433-4cff-8f6c-8052657f8c96}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
30432025-04-17T00:32:34.845ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30442025-04-17T00:32:34.845ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fd0b6216-cba6-4eea-95b1-92a28d06ce3b", "date": "Thu, 17 Apr 2025 00:32:34 GMT"} })
30452025-04-17T00:32:34.845ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30462025-04-17T00:32:34.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30472025-04-17T00:32:34.847ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30482025-04-17T00:32:34.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30492025-04-17T00:32:34.851ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30502025-04-17T00:32:34.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30512025-04-17T00:32:34.854ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30522025-04-17T00:32:34.854ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30532025-04-17T00:32:34.854ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
30542025-04-17T00:32:34.854ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30552025-04-17T00:32:34.854ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "39df5840-ce98-4910-8513-da254ca1cdd2", "date": "Thu, 17 Apr 2025 00:32:34 GMT"} })
30562025-04-17T00:32:34.856ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 71.716116ms iteration = 24 status = {}
30572025-04-17T00:32:34.858ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30582025-04-17T00:32:34.858ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30592025-04-17T00:32:34.870ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30602025-04-17T00:32:34.871ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "69835c41-c4e0-429e-bbcc-b68f37f08228", "content-length": "691", "date": "Thu, 17 Apr 2025 00:32:34 GMT"} })
30612025-04-17T00:32:34.871ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
30622025-04-17T00:32:34.871ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
30632025-04-17T00:32:34.871ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
30642025-04-17T00:32:34.872ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 87.466703ms iteration = 24 status = {}
30652025-04-17T00:32:34.874ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 propolis_id = 0eb88743-e5c4-471f-8837-aefa3728c3a9 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
30662025-04-17T00:32:34.874ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = MarkVmmDeleted
30672025-04-17T00:32:34.874ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = MarkVmmDeleted
30682025-04-17T00:32:34.885ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30692025-04-17T00:32:34.885ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30702025-04-17T00:32:34.889ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30712025-04-17T00:32:34.890ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30722025-04-17T00:32:34.893ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30732025-04-17T00:32:34.893ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30742025-04-17T00:32:34.896ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30752025-04-17T00:32:34.896ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30762025-04-17T00:32:34.899ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30772025-04-17T00:32:34.900ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30782025-04-17T00:32:34.901ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30792025-04-17T00:32:34.901ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30802025-04-17T00:32:34.903ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30812025-04-17T00:32:34.903ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30822025-04-17T00:32:34.904ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30832025-04-17T00:32:34.908ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4614ac20-ae32-44af-af3c-2b45ace5dda2, lookup_type: ById(4614ac20-ae32-44af-af3c-2b45ace5dda2) }, key: 7ca11a5c-53f9-4c19-846c-c1c64ef8797a, lookup_type: ById(7ca11a5c-53f9-4c19-846c-c1c64ef8797a) }, key: 3b862e19-829e-4073-8318-fa518ae6d138, lookup_type: ById(3b862e19-829e-4073-8318-fa518ae6d138) } result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30842025-04-17T00:32:34.908ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30852025-04-17T00:32:34.908ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_node = ChainSuccessorSaga
30862025-04-17T00:32:34.913ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 karmic_state = WillReincarnate needs_reincarnation = false saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update
30872025-04-17T00:32:34.913ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30882025-04-17T00:32:34.913ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30892025-04-17T00:32:34.916ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30902025-04-17T00:32:34.916ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30912025-04-17T00:32:34.920ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30922025-04-17T00:32:34.920ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30932025-04-17T00:32:34.922ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30942025-04-17T00:32:34.922ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a
30952025-04-17T00:32:34.931ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30962025-04-17T00:32:34.931ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a saga_name = instance-update saga_result = SagaResult { saga_id: 5fa43c76-3932-4f25-96f5-e90d77ac4b6a, saga_log: SagaLog { saga_id: 5fa43c76-3932-4f25-96f5-e90d77ac4b6a, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("47891844-9140-4f6c-809a-49c4294f8ff9")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("47891844-9140-4f6c-809a-49c4294f8ff9")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("3b862e19-829e-4073-8318-fa518ae6d138"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("0eb88743-e5c4-471f-8837-aefa3728c3a9")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("3b862e19-829e-4073-8318-fa518ae6d138"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("0eb88743-e5c4-471f-8837-aefa3728c3a9")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("47891844-9140-4f6c-809a-49c4294f8ff9"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("47891844-9140-4f6c-809a-49c4294f8ff9")}} }) }
30972025-04-17T00:32:34.931ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 5fa43c76-3932-4f25-96f5-e90d77ac4b6a instance_id = 3b862e19-829e-4073-8318-fa518ae6d138 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update
30982025-04-17T00:32:34.931ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
30992025-04-17T00:32:34.931ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
31002025-04-17T00:32:34.935ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
31012025-04-17T00:32:34.935ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
31022025-04-17T00:32:34.938ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
31032025-04-17T00:32:34.938ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
31042025-04-17T00:32:34.941ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
31052025-04-17T00:32:34.941ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070
31062025-04-17T00:32:35.001ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update sec_id = ef9eec85-7cbe-4a5f-89ba-f11462071c47
31072025-04-17T00:32:35.001ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = bf1d6dba-5697-4371-8dcd-8e114b774070 saga_name = start-instance-update saga_result = SagaResult { saga_id: bf1d6dba-5697-4371-8dcd-8e114b774070, saga_log: SagaLog { saga_id: bf1d6dba-5697-4371-8dcd-8e114b774070, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("57b98b0c-6c46-4a3c-b0f7-3894d172878f")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("57b98b0c-6c46-4a3c-b0f7-3894d172878f")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("57b98b0c-6c46-4a3c-b0f7-3894d172878f"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("57b98b0c-6c46-4a3c-b0f7-3894d172878f")}} }) }
31082025-04-17T00:32:35.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8ecddb0b-2fb2-47e1-9af2-e47d7d5ce8bd roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31092025-04-17T00:32:35.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8ecddb0b-2fb2-47e1-9af2-e47d7d5ce8bd resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31102025-04-17T00:32:35.786ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2155 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8ecddb0b-2fb2-47e1-9af2-e47d7d5ce8bd response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31112025-04-17T00:32:36.807ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 08152345-4c58-4cc2-8772-2e2e7751a9a5 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31122025-04-17T00:32:36.807ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 08152345-4c58-4cc2-8772-2e2e7751a9a5 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31132025-04-17T00:32:36.809ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2116 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 08152345-4c58-4cc2-8772-2e2e7751a9a5 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31142025-04-17T00:32:37.831ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6e6335c-8ce0-4953-a0c8-130fe5a9491a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31152025-04-17T00:32:37.831ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6e6335c-8ce0-4953-a0c8-130fe5a9491a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31162025-04-17T00:32:37.833ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2244 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6e6335c-8ce0-4953-a0c8-130fe5a9491a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31172025-04-17T00:32:37.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
31182025-04-17T00:32:37.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
31192025-04-17T00:32:37.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31202025-04-17T00:32:37.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31212025-04-17T00:32:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31222025-04-17T00:32:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31232025-04-17T00:32:37.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31242025-04-17T00:32:37.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31252025-04-17T00:32:37.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31262025-04-17T00:32:37.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31272025-04-17T00:32:37.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31282025-04-17T00:32:37.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31292025-04-17T00:32:37.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31302025-04-17T00:32:37.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31312025-04-17T00:32:37.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31322025-04-17T00:32:37.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
31332025-04-17T00:32:37.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31342025-04-17T00:32:37.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31352025-04-17T00:32:37.983ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
31362025-04-17T00:32:37.983ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.992864ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
31372025-04-17T00:32:37.996ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.305449ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31382025-04-17T00:32:38.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9c0db764-27a6-4cc1-804d-670f224a05f0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31392025-04-17T00:32:38.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9c0db764-27a6-4cc1-804d-670f224a05f0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31402025-04-17T00:32:38.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1819 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 9c0db764-27a6-4cc1-804d-670f224a05f0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31412025-04-17T00:32:39.873ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8727a7e8-5abb-4ae8-9fab-5f9e57afbec2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31422025-04-17T00:32:39.874ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8727a7e8-5abb-4ae8-9fab-5f9e57afbec2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31432025-04-17T00:32:39.876ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2649 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8727a7e8-5abb-4ae8-9fab-5f9e57afbec2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31442025-04-17T00:32:40.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e5c9e330-1086-4359-ac75-f91303da63a6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31452025-04-17T00:32:40.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e5c9e330-1086-4359-ac75-f91303da63a6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31462025-04-17T00:32:40.899ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2517 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e5c9e330-1086-4359-ac75-f91303da63a6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31472025-04-17T00:32:41.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 575 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = b05b95fd-2087-4c9f-8f98-1f1f7b78e095 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
31482025-04-17T00:32:41.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d5e08287-6dfe-499e-9268-0eae612a3cb6 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31492025-04-17T00:32:41.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d5e08287-6dfe-499e-9268-0eae612a3cb6 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31502025-04-17T00:32:41.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2562 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d5e08287-6dfe-499e-9268-0eae612a3cb6 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31512025-04-17T00:32:42.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 516b8dd1-5f19-4e15-93ae-8e0df98126cf roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31522025-04-17T00:32:42.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 516b8dd1-5f19-4e15-93ae-8e0df98126cf resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31532025-04-17T00:32:42.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2355 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 516b8dd1-5f19-4e15-93ae-8e0df98126cf response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31542025-04-17T00:32:43.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f3f8a1c6-c092-4e69-a946-40972a30cea0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31552025-04-17T00:32:43.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f3f8a1c6-c092-4e69-a946-40972a30cea0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31562025-04-17T00:32:43.965ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2177 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f3f8a1c6-c092-4e69-a946-40972a30cea0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31572025-04-17T00:32:44.984ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bbaf17b5-c5e7-440a-8645-be5a0609ce74 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31582025-04-17T00:32:44.985ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bbaf17b5-c5e7-440a-8645-be5a0609ce74 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31592025-04-17T00:32:44.987ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2282 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = bbaf17b5-c5e7-440a-8645-be5a0609ce74 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31602025-04-17T00:32:46.005ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 92e565da-4a9e-41a7-8f9f-ad2489957fd2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31612025-04-17T00:32:46.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 92e565da-4a9e-41a7-8f9f-ad2489957fd2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31622025-04-17T00:32:46.008ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2612 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 92e565da-4a9e-41a7-8f9f-ad2489957fd2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31632025-04-17T00:32:46.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 0534f956-1f9d-4da4-840d-f08d986dc2e0 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
31642025-04-17T00:32:46.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 0534f956-1f9d-4da4-840d-f08d986dc2e0 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
31652025-04-17T00:32:46.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3496 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 0534f956-1f9d-4da4-840d-f08d986dc2e0 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
31662025-04-17T00:32:46.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 2b783be1-62cf-463f-83d5-216d306000d5 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
31672025-04-17T00:32:46.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 2b783be1-62cf-463f-83d5-216d306000d5 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
31682025-04-17T00:32:46.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2818 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 2b783be1-62cf-463f-83d5-216d306000d5 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
31692025-04-17T00:32:47.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f5cc8c34-69e8-42bf-9632-67a2c176397d roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31702025-04-17T00:32:47.027ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f5cc8c34-69e8-42bf-9632-67a2c176397d resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31712025-04-17T00:32:47.029ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2175 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f5cc8c34-69e8-42bf-9632-67a2c176397d response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
31722025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
31732025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
31742025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31752025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31762025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
31772025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
31782025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
31792025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31802025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31812025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
31822025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
31832025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31842025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31852025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
31862025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
31872025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
31882025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31892025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
31902025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31912025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
31922025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
31932025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31942025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
31952025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31962025-04-17T00:32:47.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31972025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31982025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
31992025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
32002025-04-17T00:32:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
32012025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
32022025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
32032025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32042025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
32052025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32062025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
32072025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
32082025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
32092025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
32102025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32112025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32122025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
32132025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32142025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32152025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32162025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32172025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32182025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32192025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32202025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
32212025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32222025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
32232025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
32242025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32252025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32262025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32272025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32282025-04-17T00:32:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
32292025-04-17T00:32:47.964ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32302025-04-17T00:32:47.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32312025-04-17T00:32:47.965ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1893, tv_nsec: 636604435 } })
32322025-04-17T00:32:47.966ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32332025-04-17T00:32:47.966ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
32342025-04-17T00:32:47.966ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.411721ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
32352025-04-17T00:32:47.966ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1893, tv_nsec: 637208061 } })
32362025-04-17T00:32:47.966ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
32372025-04-17T00:32:47.966ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32382025-04-17T00:32:47.966ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
32392025-04-17T00:32:47.966ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32402025-04-17T00:32:47.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32412025-04-17T00:32:47.967ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32422025-04-17T00:32:47.967ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32432025-04-17T00:32:47.968ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.635637ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
32442025-04-17T00:32:47.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32452025-04-17T00:32:47.968ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32462025-04-17T00:32:47.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32472025-04-17T00:32:47.970ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32482025-04-17T00:32:47.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32492025-04-17T00:32:47.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32502025-04-17T00:32:47.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32512025-04-17T00:32:47.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32522025-04-17T00:32:47.971ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32532025-04-17T00:32:47.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32542025-04-17T00:32:47.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32552025-04-17T00:32:47.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32562025-04-17T00:32:47.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32572025-04-17T00:32:47.978ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32582025-04-17T00:32:47.979ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32592025-04-17T00:32:47.979ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 17.421049ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
32602025-04-17T00:32:47.980ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32612025-04-17T00:32:47.980ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32622025-04-17T00:32:47.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32632025-04-17T00:32:47.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
32642025-04-17T00:32:47.982ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32652025-04-17T00:32:47.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32662025-04-17T00:32:47.987ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32672025-04-17T00:32:47.987ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1893, tv_nsec: 658113557 } })
32682025-04-17T00:32:47.987ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
32692025-04-17T00:32:47.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32702025-04-17T00:32:47.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32712025-04-17T00:32:47.989ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32722025-04-17T00:32:47.989ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32732025-04-17T00:32:47.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32742025-04-17T00:32:47.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32752025-04-17T00:32:47.990ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32762025-04-17T00:32:47.990ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32772025-04-17T00:32:47.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32782025-04-17T00:32:47.992ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32792025-04-17T00:32:47.995ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32802025-04-17T00:32:47.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32812025-04-17T00:32:47.995ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32822025-04-17T00:32:47.995ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32832025-04-17T00:32:47.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32842025-04-17T00:32:48.018ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 56.75607ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
32852025-04-17T00:32:48.021ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32862025-04-17T00:32:48.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32872025-04-17T00:32:48.024ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32882025-04-17T00:32:48.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32892025-04-17T00:32:48.026ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32902025-04-17T00:32:48.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32912025-04-17T00:32:48.028ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
32922025-04-17T00:32:48.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32932025-04-17T00:32:48.028ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32942025-04-17T00:32:48.031ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
32952025-04-17T00:32:48.031ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32962025-04-17T00:32:48.031ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32972025-04-17T00:32:48.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32982025-04-17T00:32:48.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32992025-04-17T00:32:48.033ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 71.85182ms iteration = 24 status = 0
33002025-04-17T00:32:48.034ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33012025-04-17T00:32:48.037ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
33022025-04-17T00:32:48.037ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33032025-04-17T00:32:48.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
33042025-04-17T00:32:48.038ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33052025-04-17T00:32:48.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
33062025-04-17T00:32:48.038ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33072025-04-17T00:32:48.038ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
33082025-04-17T00:32:48.038ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33092025-04-17T00:32:48.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33102025-04-17T00:32:48.038ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33112025-04-17T00:32:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33122025-04-17T00:32:48.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33132025-04-17T00:32:48.045ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
33142025-04-17T00:32:48.045ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 83.795631ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
33152025-04-17T00:32:48.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
33162025-04-17T00:32:48.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33172025-04-17T00:32:48.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33182025-04-17T00:32:48.047ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
33192025-04-17T00:32:48.047ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33202025-04-17T00:32:48.047ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9929971-7491-455f-803d-c5db9d82394e", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
33212025-04-17T00:32:48.047ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33222025-04-17T00:32:48.049ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33232025-04-17T00:32:48.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b716d810-ab3a-48eb-b0f6-12439b662fa3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
33242025-04-17T00:32:48.053ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b716d810-ab3a-48eb-b0f6-12439b662fa3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
33252025-04-17T00:32:48.057ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4834 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b716d810-ab3a-48eb-b0f6-12439b662fa3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
33262025-04-17T00:32:48.063ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 101.762263ms iteration = 27 status = {"physical_disks_added":0}
33272025-04-17T00:32:48.065ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
33282025-04-17T00:32:48.065ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 103.61711ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
33292025-04-17T00:32:48.066ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 104.579274ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33302025-04-17T00:32:48.072ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
33312025-04-17T00:32:48.072ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33322025-04-17T00:32:48.072ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f8c4cb2-57c3-45bf-baee-98fdd4884732", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
33332025-04-17T00:32:48.072ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33342025-04-17T00:32:48.074ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 112.314909ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
33352025-04-17T00:32:48.074ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 113.063868ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
33362025-04-17T00:32:48.075ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33372025-04-17T00:32:48.082ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33382025-04-17T00:32:48.082ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
33392025-04-17T00:32:48.082ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 120.409843ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
33402025-04-17T00:32:48.087ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 125.85541ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
33412025-04-17T00:32:48.089ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33422025-04-17T00:32:48.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33432025-04-17T00:32:48.091ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33442025-04-17T00:32:48.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33452025-04-17T00:32:48.093ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
33462025-04-17T00:32:48.093ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 131.545303ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
33472025-04-17T00:32:48.110ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
33482025-04-17T00:32:48.110ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33492025-04-17T00:32:48.111ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8b69864-c840-4a6e-b9fc-e15ae41cf536", "content-length": "26", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
33502025-04-17T00:32:48.111ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33512025-04-17T00:32:48.112ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
33522025-04-17T00:32:48.126ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33532025-04-17T00:32:48.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33542025-04-17T00:32:48.182ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 220.925895ms iteration = 24 status = {}
33552025-04-17T00:32:48.191ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
33562025-04-17T00:32:48.191ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33572025-04-17T00:32:48.192ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33582025-04-17T00:32:48.192ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8714f0d0-3c36-4441-92b9-5f92cc9c9247", "content-length": "372", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
33592025-04-17T00:32:48.192ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33602025-04-17T00:32:48.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33612025-04-17T00:32:48.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33622025-04-17T00:32:48.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33632025-04-17T00:32:48.195ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33642025-04-17T00:32:48.195ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33652025-04-17T00:32:48.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33662025-04-17T00:32:48.197ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 235.682657ms iteration = 24 status = null
33672025-04-17T00:32:48.197ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33682025-04-17T00:32:48.199ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33692025-04-17T00:32:48.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33702025-04-17T00:32:48.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33712025-04-17T00:32:48.237ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
33722025-04-17T00:32:48.237ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33732025-04-17T00:32:48.237ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "795200ad-9cf2-4893-9c9e-154d7828735b", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
33742025-04-17T00:32:48.237ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33752025-04-17T00:32:48.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33762025-04-17T00:32:48.240ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33772025-04-17T00:32:48.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33782025-04-17T00:32:48.243ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33792025-04-17T00:32:48.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33802025-04-17T00:32:48.247ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
33812025-04-17T00:32:48.247ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33822025-04-17T00:32:48.264ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
33832025-04-17T00:32:48.305ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33842025-04-17T00:32:48.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33852025-04-17T00:32:48.308ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
33862025-04-17T00:32:48.308ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33872025-04-17T00:32:48.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33882025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33892025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
33902025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
33912025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33922025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33932025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33942025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33952025-04-17T00:32:48.318ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33962025-04-17T00:32:48.319ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33972025-04-17T00:32:48.319ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33982025-04-17T00:32:48.319ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
33992025-04-17T00:32:48.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34002025-04-17T00:32:48.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34012025-04-17T00:32:48.322ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
34022025-04-17T00:32:48.322ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34032025-04-17T00:32:48.359ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1f72b058-491d-416f-8efc-f5a822242a76", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
34042025-04-17T00:32:48.360ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 398.57044ms iteration = 25 status = {}
34052025-04-17T00:32:48.364ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac61afb3-1db3-4ee4-9c54-69808d046f0e", "content-length": "691", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
34062025-04-17T00:32:48.364ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
34072025-04-17T00:32:48.364ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
34082025-04-17T00:32:48.364ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
34092025-04-17T00:32:48.365ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 404.189421ms iteration = 25 status = {}
34102025-04-17T00:32:48.464ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2f75a284-2dd2-4014-ab2c-8b134c31879b", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
34112025-04-17T00:32:48.464ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
34122025-04-17T00:32:48.464ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34132025-04-17T00:32:48.468ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f37cb84-569d-4884-9505-7922709b4dda", "content-length": "682", "date": "Thu, 17 Apr 2025 00:32:48 GMT"} })
34142025-04-17T00:32:48.468ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34152025-04-17T00:32:48.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34162025-04-17T00:32:48.472ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34172025-04-17T00:32:48.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34182025-04-17T00:32:48.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34192025-04-17T00:32:48.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34202025-04-17T00:32:48.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34212025-04-17T00:32:48.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34222025-04-17T00:32:48.486ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 524.778758ms iteration = 24 status = {}
34232025-04-17T00:32:49.081ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e45f547f-8f7b-4cf0-9ad2-8d6b68a82d2f roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34242025-04-17T00:32:49.082ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e45f547f-8f7b-4cf0-9ad2-8d6b68a82d2f resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34252025-04-17T00:32:49.084ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2412 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e45f547f-8f7b-4cf0-9ad2-8d6b68a82d2f response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34262025-04-17T00:32:50.103ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e2095aae-4055-4af9-a57e-5cf84e4c6e61 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34272025-04-17T00:32:50.103ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e2095aae-4055-4af9-a57e-5cf84e4c6e61 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34282025-04-17T00:32:50.105ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2286 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e2095aae-4055-4af9-a57e-5cf84e4c6e61 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34292025-04-17T00:32:51.124ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 26cb58b1-38b7-4ca8-8e5e-75c93012ca87 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34302025-04-17T00:32:51.124ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 26cb58b1-38b7-4ca8-8e5e-75c93012ca87 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34312025-04-17T00:32:51.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2027 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 26cb58b1-38b7-4ca8-8e5e-75c93012ca87 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34322025-04-17T00:32:51.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 632 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 0070b86a-0117-4206-ba61-e60dfc41eceb response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
34332025-04-17T00:32:52.147ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fb403a59-f85b-4650-b38f-5e0b7f8acd80 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34342025-04-17T00:32:52.147ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fb403a59-f85b-4650-b38f-5e0b7f8acd80 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34352025-04-17T00:32:52.149ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2513 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fb403a59-f85b-4650-b38f-5e0b7f8acd80 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34362025-04-17T00:32:53.170ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a323aa6f-d92d-4460-8d2d-f747f8ce4d57 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34372025-04-17T00:32:53.170ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a323aa6f-d92d-4460-8d2d-f747f8ce4d57 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34382025-04-17T00:32:53.172ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2332 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a323aa6f-d92d-4460-8d2d-f747f8ce4d57 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34392025-04-17T00:32:54.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c2ec1d98-8037-414b-b79b-105a6acdc653 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34402025-04-17T00:32:54.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c2ec1d98-8037-414b-b79b-105a6acdc653 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34412025-04-17T00:32:54.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2372 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c2ec1d98-8037-414b-b79b-105a6acdc653 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34422025-04-17T00:32:55.214ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8b1e8b43-253d-450d-99f8-05577b97faa0 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34432025-04-17T00:32:55.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8b1e8b43-253d-450d-99f8-05577b97faa0 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34442025-04-17T00:32:55.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2587 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 8b1e8b43-253d-450d-99f8-05577b97faa0 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34452025-04-17T00:32:56.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2ea16f3a-b08a-4004-a55a-2bda1546b94b roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34462025-04-17T00:32:56.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2ea16f3a-b08a-4004-a55a-2bda1546b94b resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34472025-04-17T00:32:56.239ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1863 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2ea16f3a-b08a-4004-a55a-2bda1546b94b response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34482025-04-17T00:32:57.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5463e640-0600-4142-922c-d2e52d49615d roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34492025-04-17T00:32:57.259ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5463e640-0600-4142-922c-d2e52d49615d resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34502025-04-17T00:32:57.261ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2709 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5463e640-0600-4142-922c-d2e52d49615d response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34512025-04-17T00:32:57.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
34522025-04-17T00:32:57.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
34532025-04-17T00:32:57.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34542025-04-17T00:32:57.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34552025-04-17T00:32:57.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34562025-04-17T00:32:57.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34572025-04-17T00:32:57.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34582025-04-17T00:32:57.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34592025-04-17T00:32:57.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34602025-04-17T00:32:57.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34612025-04-17T00:32:57.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34622025-04-17T00:32:57.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34632025-04-17T00:32:57.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34642025-04-17T00:32:57.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
34652025-04-17T00:32:57.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34662025-04-17T00:32:57.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34672025-04-17T00:32:57.981ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
34682025-04-17T00:32:57.981ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.324163ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
34692025-04-17T00:32:57.991ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34702025-04-17T00:32:57.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34712025-04-17T00:32:57.993ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.981786ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34722025-04-17T00:32:58.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6ef7d56-fd12-47f0-a220-2a8bf2285ecc roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34732025-04-17T00:32:58.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6ef7d56-fd12-47f0-a220-2a8bf2285ecc resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34742025-04-17T00:32:58.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1839 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d6ef7d56-fd12-47f0-a220-2a8bf2285ecc response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34752025-04-17T00:32:59.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cdbf0f95-033e-47fa-9f2a-ab1bdc660f32 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34762025-04-17T00:32:59.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cdbf0f95-033e-47fa-9f2a-ab1bdc660f32 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34772025-04-17T00:32:59.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2509 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = cdbf0f95-033e-47fa-9f2a-ab1bdc660f32 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34782025-04-17T00:33:00.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5325e970-1683-43ea-8094-4ae877c243b1 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34792025-04-17T00:33:00.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5325e970-1683-43ea-8094-4ae877c243b1 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34802025-04-17T00:33:00.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1944 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5325e970-1683-43ea-8094-4ae877c243b1 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34812025-04-17T00:33:01.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 63c0e8bb-f044-404f-8e90-35bbdd46d50c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34822025-04-17T00:33:01.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 63c0e8bb-f044-404f-8e90-35bbdd46d50c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34832025-04-17T00:33:01.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2576 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 63c0e8bb-f044-404f-8e90-35bbdd46d50c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34842025-04-17T00:33:01.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 707 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = cba4c209-c866-4208-bc4c-83b24562e7af response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
34852025-04-17T00:33:02.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1435f459-f8cf-4175-ab15-1b28191d9f5a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34862025-04-17T00:33:02.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1435f459-f8cf-4175-ab15-1b28191d9f5a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34872025-04-17T00:33:02.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2661 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1435f459-f8cf-4175-ab15-1b28191d9f5a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34882025-04-17T00:33:03.388ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 734ac177-7220-49bc-bcd3-7db847102f40 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34892025-04-17T00:33:03.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 734ac177-7220-49bc-bcd3-7db847102f40 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34902025-04-17T00:33:03.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2184 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 734ac177-7220-49bc-bcd3-7db847102f40 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34912025-04-17T00:33:04.410ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 427bf2ca-a7a2-4380-8b8f-ac7e3a0993d7 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34922025-04-17T00:33:04.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 427bf2ca-a7a2-4380-8b8f-ac7e3a0993d7 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34932025-04-17T00:33:04.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2461 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 427bf2ca-a7a2-4380-8b8f-ac7e3a0993d7 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34942025-04-17T00:33:05.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5e230d8b-9696-4bf2-b79c-26b152bb0e6c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34952025-04-17T00:33:05.433ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5e230d8b-9696-4bf2-b79c-26b152bb0e6c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34962025-04-17T00:33:05.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2184 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 5e230d8b-9696-4bf2-b79c-26b152bb0e6c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34972025-04-17T00:33:06.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 300af830-39d6-44d2-85d9-e495bdde37f2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34982025-04-17T00:33:06.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 300af830-39d6-44d2-85d9-e495bdde37f2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
34992025-04-17T00:33:06.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2982 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 300af830-39d6-44d2-85d9-e495bdde37f2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35002025-04-17T00:33:07.474ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 35a332eb-1d7f-4228-9a02-3b7a05ffa729 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35012025-04-17T00:33:07.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 35a332eb-1d7f-4228-9a02-3b7a05ffa729 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35022025-04-17T00:33:07.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2947 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 35a332eb-1d7f-4228-9a02-3b7a05ffa729 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35032025-04-17T00:33:07.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
35042025-04-17T00:33:07.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
35052025-04-17T00:33:07.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35062025-04-17T00:33:07.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35072025-04-17T00:33:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35082025-04-17T00:33:07.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35092025-04-17T00:33:07.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35102025-04-17T00:33:07.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35112025-04-17T00:33:07.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35122025-04-17T00:33:07.965ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.96299ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35132025-04-17T00:33:07.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35142025-04-17T00:33:07.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35152025-04-17T00:33:07.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35162025-04-17T00:33:07.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35172025-04-17T00:33:07.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35182025-04-17T00:33:07.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35192025-04-17T00:33:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
35202025-04-17T00:33:07.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35212025-04-17T00:33:07.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35222025-04-17T00:33:07.978ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
35232025-04-17T00:33:07.978ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.241174ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
35242025-04-17T00:33:08.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = af271cdd-af5a-4cae-a991-05cde575d4ea roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35252025-04-17T00:33:08.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = af271cdd-af5a-4cae-a991-05cde575d4ea resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35262025-04-17T00:33:08.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = af271cdd-af5a-4cae-a991-05cde575d4ea response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35272025-04-17T00:33:09.517ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 904c6ede-647e-49b6-9605-813056ec06b3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35282025-04-17T00:33:09.517ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 904c6ede-647e-49b6-9605-813056ec06b3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35292025-04-17T00:33:09.519ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 904c6ede-647e-49b6-9605-813056ec06b3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35302025-04-17T00:33:10.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 668300b7-4158-4641-958f-8b93b2ff1e33 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35312025-04-17T00:33:10.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 668300b7-4158-4641-958f-8b93b2ff1e33 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35322025-04-17T00:33:10.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2821 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 668300b7-4158-4641-958f-8b93b2ff1e33 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35332025-04-17T00:33:11.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e26e42ff-61b4-4fc5-9cf5-0e66ccb20d90 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35342025-04-17T00:33:11.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e26e42ff-61b4-4fc5-9cf5-0e66ccb20d90 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35352025-04-17T00:33:11.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2366 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e26e42ff-61b4-4fc5-9cf5-0e66ccb20d90 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35362025-04-17T00:33:11.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 780 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = fdc54178-4f1c-48e6-a509-1d7c0d1265d8 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
35372025-04-17T00:33:12.583ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7e18a4b2-bce4-4052-ad77-675db004e945 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35382025-04-17T00:33:12.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7e18a4b2-bce4-4052-ad77-675db004e945 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35392025-04-17T00:33:12.585ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2163 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7e18a4b2-bce4-4052-ad77-675db004e945 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35402025-04-17T00:33:13.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4007d48a-f10a-419f-b020-108552f2e9c3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35412025-04-17T00:33:13.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4007d48a-f10a-419f-b020-108552f2e9c3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35422025-04-17T00:33:13.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2443 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 4007d48a-f10a-419f-b020-108552f2e9c3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35432025-04-17T00:33:14.626ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 82f5481a-36fd-4aa2-a5d2-cc51ce097d84 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35442025-04-17T00:33:14.626ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 82f5481a-36fd-4aa2-a5d2-cc51ce097d84 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35452025-04-17T00:33:14.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2094 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 82f5481a-36fd-4aa2-a5d2-cc51ce097d84 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35462025-04-17T00:33:15.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ff7d80ac-1910-45df-934d-5fa3a40cd544 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35472025-04-17T00:33:15.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ff7d80ac-1910-45df-934d-5fa3a40cd544 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35482025-04-17T00:33:15.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2390 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ff7d80ac-1910-45df-934d-5fa3a40cd544 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35492025-04-17T00:33:16.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = af0df143-82c5-4c74-9e95-302c83f9cb38 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
35502025-04-17T00:33:16.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = af0df143-82c5-4c74-9e95-302c83f9cb38 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
35512025-04-17T00:33:16.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3694 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = af0df143-82c5-4c74-9e95-302c83f9cb38 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
35522025-04-17T00:33:16.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 6b27d44b-763c-49a2-8cf7-48bfbd52b838 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
35532025-04-17T00:33:16.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 6b27d44b-763c-49a2-8cf7-48bfbd52b838 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
35542025-04-17T00:33:16.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2848 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 6b27d44b-763c-49a2-8cf7-48bfbd52b838 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
35552025-04-17T00:33:16.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 03b00821-866c-4382-8ea1-ac73faf84e5a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35562025-04-17T00:33:16.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 03b00821-866c-4382-8ea1-ac73faf84e5a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35572025-04-17T00:33:16.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2443 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 03b00821-866c-4382-8ea1-ac73faf84e5a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35582025-04-17T00:33:17.693ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 58e13914-df02-4182-bbfe-5108b0218aae roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35592025-04-17T00:33:17.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 58e13914-df02-4182-bbfe-5108b0218aae resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35602025-04-17T00:33:17.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2243 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 58e13914-df02-4182-bbfe-5108b0218aae response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
35612025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
35622025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
35632025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
35642025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
35652025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35662025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35672025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
35682025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
35692025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35702025-04-17T00:33:17.961ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-17 00:23:17.961795515 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
35712025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
35722025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
35732025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
35742025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35752025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
35762025-04-17T00:33:17.961ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3
35772025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.43\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3"}
35782025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
35792025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
35802025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35812025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
35822025-04-17T00:33:17.961ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35832025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
35842025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35852025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
35862025-04-17T00:33:17.962ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
35872025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
35882025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
35892025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
35902025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
35912025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
35922025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
35932025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35942025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
35952025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
35962025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35972025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
35982025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35992025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
36002025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36012025-04-17T00:33:17.962ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
36022025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36032025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
36042025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
36052025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
36062025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36072025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
36082025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
36092025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
36102025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36112025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
36122025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
36132025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
36142025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
36152025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
36162025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
36172025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
36182025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
36192025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36202025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
36212025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
36222025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
36232025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36242025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
36252025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
36262025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36272025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36282025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
36292025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
36302025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
36312025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36322025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36332025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
36342025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36352025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36362025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36372025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
36382025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
36392025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
36402025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36412025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36422025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
36432025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36442025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36452025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36462025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36472025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36482025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36492025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
36502025-04-17T00:33:17.962ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b3e44f3-f539-46b4-bb67-d442c7ebd3cf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4488d95c-7f86-4ca6-ab72-2e345a6a57bb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("777fc2e9-7e08-4efb-9297-7184477cc904.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1923, tv_nsec: 633301887 } })
36512025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
36522025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36532025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
36542025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36552025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36562025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36572025-04-17T00:33:17.962ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("99145431-02b3-4b6e-aa5d-968f96332449.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c4d82657-8205-4ac8-b622-739dd8e6c53f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1923, tv_nsec: 633370669 } })
36582025-04-17T00:33:17.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36592025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36602025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36612025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36622025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36632025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
36642025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.296413ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
36652025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
36662025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.387455ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
36672025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36682025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36692025-04-17T00:33:17.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36702025-04-17T00:33:17.964ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36712025-04-17T00:33:17.965ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1923, tv_nsec: 636096138 } })
36722025-04-17T00:33:17.965ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
36732025-04-17T00:33:17.966ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.313749ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36742025-04-17T00:33:17.966ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-17 00:23:17.961795515 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
36752025-04-17T00:33:17.966ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.760279ms iteration = 13 status = {"expiration":"2025-04-17T00:23:17.961795515Z","pruned":[]}
36762025-04-17T00:33:17.966ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
36772025-04-17T00:33:17.966ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36782025-04-17T00:33:17.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
36792025-04-17T00:33:17.968ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36802025-04-17T00:33:17.968ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
36812025-04-17T00:33:17.968ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36822025-04-17T00:33:17.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36832025-04-17T00:33:17.968ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36842025-04-17T00:33:17.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36852025-04-17T00:33:17.969ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1923, tv_nsec: 639847522 } })
36862025-04-17T00:33:17.969ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
36872025-04-17T00:33:17.974ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.524336ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
36882025-04-17T00:33:17.974ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36892025-04-17T00:33:17.976ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36902025-04-17T00:33:17.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
36912025-04-17T00:33:17.978ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
36922025-04-17T00:33:17.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
36932025-04-17T00:33:17.979ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36942025-04-17T00:33:17.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
36952025-04-17T00:33:17.979ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36962025-04-17T00:33:17.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36972025-04-17T00:33:17.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36982025-04-17T00:33:17.981ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36992025-04-17T00:33:17.988ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37002025-04-17T00:33:17.988ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37012025-04-17T00:33:17.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37022025-04-17T00:33:17.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37032025-04-17T00:33:17.989ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37042025-04-17T00:33:17.990ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37052025-04-17T00:33:17.991ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 29.609666ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
37062025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37072025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37082025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37092025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
37102025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37112025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
37122025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37132025-04-17T00:33:17.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37142025-04-17T00:33:17.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37152025-04-17T00:33:17.994ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37162025-04-17T00:33:17.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37172025-04-17T00:33:17.998ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
37182025-04-17T00:33:17.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
37192025-04-17T00:33:18.000ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37202025-04-17T00:33:18.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37212025-04-17T00:33:18.003ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37222025-04-17T00:33:18.004ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37232025-04-17T00:33:18.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
37242025-04-17T00:33:18.007ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37252025-04-17T00:33:18.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37262025-04-17T00:33:18.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37272025-04-17T00:33:18.007ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37282025-04-17T00:33:18.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37292025-04-17T00:33:18.009ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37302025-04-17T00:33:18.010ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
37312025-04-17T00:33:18.010ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
37322025-04-17T00:33:18.010ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 48.44792ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
37332025-04-17T00:33:18.012ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37342025-04-17T00:33:18.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
37352025-04-17T00:33:18.013ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37362025-04-17T00:33:18.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37372025-04-17T00:33:18.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37382025-04-17T00:33:18.013ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37392025-04-17T00:33:18.014ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37402025-04-17T00:33:18.014ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
37412025-04-17T00:33:18.014ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37422025-04-17T00:33:18.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37432025-04-17T00:33:18.016ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
37442025-04-17T00:33:18.016ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37452025-04-17T00:33:18.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37462025-04-17T00:33:18.016ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37472025-04-17T00:33:18.018ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
37482025-04-17T00:33:18.018ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37492025-04-17T00:33:18.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37502025-04-17T00:33:18.018ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37512025-04-17T00:33:18.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37522025-04-17T00:33:18.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37532025-04-17T00:33:18.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37542025-04-17T00:33:18.023ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1923, tv_nsec: 693898674 } })
37552025-04-17T00:33:18.023ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37562025-04-17T00:33:18.023ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
37572025-04-17T00:33:18.024ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
37582025-04-17T00:33:18.024ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 62.461553ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
37592025-04-17T00:33:18.025ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37602025-04-17T00:33:18.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37612025-04-17T00:33:18.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
37622025-04-17T00:33:18.034ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.238849ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
37632025-04-17T00:33:18.035ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37642025-04-17T00:33:18.036ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37652025-04-17T00:33:18.037ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37662025-04-17T00:33:18.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
37672025-04-17T00:33:18.038ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37682025-04-17T00:33:18.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37692025-04-17T00:33:18.039ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 77.41641ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
37702025-04-17T00:33:18.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
37712025-04-17T00:33:18.040ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37722025-04-17T00:33:18.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37732025-04-17T00:33:18.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
37742025-04-17T00:33:18.040ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37752025-04-17T00:33:18.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37762025-04-17T00:33:18.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37772025-04-17T00:33:18.044ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37782025-04-17T00:33:18.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37792025-04-17T00:33:18.046ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 84.285513ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
37802025-04-17T00:33:18.046ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37812025-04-17T00:33:18.048ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
37822025-04-17T00:33:18.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37832025-04-17T00:33:18.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37842025-04-17T00:33:18.050ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37852025-04-17T00:33:18.050ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37862025-04-17T00:33:18.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37872025-04-17T00:33:18.054ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 93.131815ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37882025-04-17T00:33:18.054ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37892025-04-17T00:33:18.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37902025-04-17T00:33:18.059ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 97.68202ms iteration = 28 status = {"physical_disks_added":0}
37912025-04-17T00:33:18.062ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
37922025-04-17T00:33:18.062ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
37932025-04-17T00:33:18.062ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 100.291356ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
37942025-04-17T00:33:18.064ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
37952025-04-17T00:33:18.064ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 102.207764ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
37962025-04-17T00:33:18.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37972025-04-17T00:33:18.068ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37982025-04-17T00:33:18.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37992025-04-17T00:33:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
38002025-04-17T00:33:18.071ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38012025-04-17T00:33:18.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38022025-04-17T00:33:18.073ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38032025-04-17T00:33:18.074ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 112.213567ms iteration = 6 status = {"nsuccess":5}
38042025-04-17T00:33:18.076ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38052025-04-17T00:33:18.077ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
38062025-04-17T00:33:18.077ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
38072025-04-17T00:33:18.077ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
38082025-04-17T00:33:18.077ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38092025-04-17T00:33:18.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38102025-04-17T00:33:18.077ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38112025-04-17T00:33:18.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38122025-04-17T00:33:18.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38132025-04-17T00:33:18.079ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38142025-04-17T00:33:18.079ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38152025-04-17T00:33:18.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38162025-04-17T00:33:18.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38172025-04-17T00:33:18.082ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38182025-04-17T00:33:18.083ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38192025-04-17T00:33:18.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38202025-04-17T00:33:18.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38212025-04-17T00:33:18.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38222025-04-17T00:33:18.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38232025-04-17T00:33:18.085ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38242025-04-17T00:33:18.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38252025-04-17T00:33:18.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38262025-04-17T00:33:18.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38272025-04-17T00:33:18.087ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38282025-04-17T00:33:18.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38292025-04-17T00:33:18.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38302025-04-17T00:33:18.087ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38312025-04-17T00:33:18.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38322025-04-17T00:33:18.092ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38332025-04-17T00:33:18.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38342025-04-17T00:33:18.092ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38352025-04-17T00:33:18.092ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
38362025-04-17T00:33:18.092ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 131.046601ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
38372025-04-17T00:33:18.094ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38382025-04-17T00:33:18.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38392025-04-17T00:33:18.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38402025-04-17T00:33:18.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38412025-04-17T00:33:18.096ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38422025-04-17T00:33:18.097ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
38432025-04-17T00:33:18.097ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 135.627926ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
38442025-04-17T00:33:18.097ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38452025-04-17T00:33:18.099ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 137.181136ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
38462025-04-17T00:33:18.100ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
38472025-04-17T00:33:18.100ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38482025-04-17T00:33:18.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38492025-04-17T00:33:18.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
38502025-04-17T00:33:18.102ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38512025-04-17T00:33:18.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38522025-04-17T00:33:18.105ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 143.596407ms iteration = 25 status = 0
38532025-04-17T00:33:18.107ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-17T00:21:15.858908Z, time_modified: 2025-04-17T00:21:15.858908Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, db_silo: Silo { identity: SiloIdentity { id: 4614ac20-ae32-44af-af3c-2b45ace5dda2, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-17T00:21:19.295428Z, time_modified: 2025-04-17T00:21:19.295428Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6" }] }) }
38542025-04-17T00:33:18.107ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 145.365352ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]}},"default_endpoint":{"silo_id":"4614ac20-ae32-44af-af3c-2b45ace5dda2","tls_certs":["1229723c7e8ef98fba2545a199865b84b42c1c920bea03b284da0d2b42a9e3a6"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
38552025-04-17T00:33:18.111ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 149.534596ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
38562025-04-17T00:33:18.124ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
38572025-04-17T00:33:18.133ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
38582025-04-17T00:33:18.133ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
38592025-04-17T00:33:18.133ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
38602025-04-17T00:33:18.133ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38612025-04-17T00:33:18.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38622025-04-17T00:33:18.136ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38632025-04-17T00:33:18.137ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
38642025-04-17T00:33:18.137ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38652025-04-17T00:33:18.137ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc3d2e38-adf8-4bb1-848d-f22894cd239a", "content-length": "26", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
38662025-04-17T00:33:18.137ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38672025-04-17T00:33:18.139ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
38682025-04-17T00:33:18.139ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 0 version = 2
38692025-04-17T00:33:18.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38702025-04-17T00:33:18.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38712025-04-17T00:33:18.139ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
38722025-04-17T00:33:18.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38732025-04-17T00:33:18.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38742025-04-17T00:33:18.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38752025-04-17T00:33:18.145ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
38762025-04-17T00:33:18.145ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 183.916963ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
38772025-04-17T00:33:18.181ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
38782025-04-17T00:33:18.181ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38792025-04-17T00:33:18.181ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2764a8fa-0e52-428a-a58b-5a9a8dbcf7e7", "content-length": "26", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
38802025-04-17T00:33:18.181ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38812025-04-17T00:33:18.184ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
38822025-04-17T00:33:18.239ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
38832025-04-17T00:33:18.335ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
38842025-04-17T00:33:18.335ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38852025-04-17T00:33:18.336ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "412ba673-7ede-4c73-a9aa-39906f115eb2", "content-length": "372", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
38862025-04-17T00:33:18.336ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38872025-04-17T00:33:18.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38882025-04-17T00:33:18.337ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38892025-04-17T00:33:18.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38902025-04-17T00:33:18.341ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38912025-04-17T00:33:18.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38922025-04-17T00:33:18.357ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
38932025-04-17T00:33:18.358ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2769bd57-2467-4dd6-8a94-955d07051ff9", "content-length": "691", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
38942025-04-17T00:33:18.358ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
38952025-04-17T00:33:18.358ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
38962025-04-17T00:33:18.358ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
38972025-04-17T00:33:18.359ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
38982025-04-17T00:33:18.359ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 397.975495ms iteration = 26 status = {}
38992025-04-17T00:33:18.359ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b9bc9f66-366d-4352-b8be-15f01fc80b1a", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39002025-04-17T00:33:18.359ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39012025-04-17T00:33:18.361ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d4f29ccb-718d-4915-a02d-3326cc424f02", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39022025-04-17T00:33:18.361ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39032025-04-17T00:33:18.362ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e121a1f9-8436-43f3-bb99-9e57d1160124 dns_zone_name = oxide.test found_so_far = 1 version = 2
39042025-04-17T00:33:18.362ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
39052025-04-17T00:33:18.362ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-17 00:21:19.348689 UTC dns_group = external generation = 2
39062025-04-17T00:33:18.362ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 399.896752ms iteration = 13 status = {"generation":2}
39072025-04-17T00:33:18.364ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39082025-04-17T00:33:18.364ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 402.345484ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
39092025-04-17T00:33:18.373ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
39102025-04-17T00:33:18.373ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
39112025-04-17T00:33:18.373ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0b1c1501-986a-4835-8d52-cd8b64a8ecf3", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
39122025-04-17T00:33:18.373ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39132025-04-17T00:33:18.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39142025-04-17T00:33:18.375ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39152025-04-17T00:33:18.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39162025-04-17T00:33:18.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39172025-04-17T00:33:18.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39182025-04-17T00:33:18.380ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
39192025-04-17T00:33:18.380ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
39202025-04-17T00:33:18.380ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39212025-04-17T00:33:18.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39222025-04-17T00:33:18.382ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39232025-04-17T00:33:18.383ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
39242025-04-17T00:33:18.383ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39252025-04-17T00:33:18.383ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "01cb077c-abae-48b0-8158-41b1f375ed5c", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
39262025-04-17T00:33:18.384ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 422.905592ms iteration = 25 status = {}
39272025-04-17T00:33:18.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39282025-04-17T00:33:18.385ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39292025-04-17T00:33:18.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39302025-04-17T00:33:18.386ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 423.961779ms iteration = 26 status = {}
39312025-04-17T00:33:18.388ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 87baf62c-cd2f-4853-aa49-6ee699b978de dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
39322025-04-17T00:33:18.388ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
39332025-04-17T00:33:18.388ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-17 00:21:18.231041 UTC dns_group = internal generation = 1
39342025-04-17T00:33:18.388ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 426.214665ms iteration = 13 status = {"generation":1}
39352025-04-17T00:33:18.408ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
39362025-04-17T00:33:18.408ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39372025-04-17T00:33:18.408ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8d1fc7b-512b-414a-9067-cfd1dbb60539", "content-length": "26", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
39382025-04-17T00:33:18.408ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39392025-04-17T00:33:18.409ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
39402025-04-17T00:33:18.495ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39412025-04-17T00:33:18.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39422025-04-17T00:33:18.497ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 535.94717ms iteration = 25 status = null
39432025-04-17T00:33:18.530ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
39442025-04-17T00:33:18.643ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
39452025-04-17T00:33:18.644ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2268b83b-ee95-4125-aa8e-5039327c0a0f", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39462025-04-17T00:33:18.644ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39472025-04-17T00:33:18.645ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d480108d-acc6-4208-ab71-fc7e9298d8f1", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39482025-04-17T00:33:18.645ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39492025-04-17T00:33:18.647ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "30ac78a6-0170-4900-b633-ba479985c692", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39502025-04-17T00:33:18.647ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39512025-04-17T00:33:18.649ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39522025-04-17T00:33:18.649ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 687.582369ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
39532025-04-17T00:33:18.689ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39542025-04-17T00:33:18.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39552025-04-17T00:33:18.691ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
39562025-04-17T00:33:18.691ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39572025-04-17T00:33:18.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39582025-04-17T00:33:18.702ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
39592025-04-17T00:33:18.702ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
39602025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
39612025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
39622025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39632025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39642025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39652025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
39662025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39672025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
39682025-04-17T00:33:18.703ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
39692025-04-17T00:33:18.703ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39702025-04-17T00:33:18.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39712025-04-17T00:33:18.706ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
39722025-04-17T00:33:18.706ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
39732025-04-17T00:33:18.763ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f0c738cb-3a6f-4c6d-836a-02caa5a7e8cb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39742025-04-17T00:33:18.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f0c738cb-3a6f-4c6d-836a-02caa5a7e8cb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39752025-04-17T00:33:18.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1593 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f0c738cb-3a6f-4c6d-836a-02caa5a7e8cb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39762025-04-17T00:33:18.856ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2257b064-ae27-4169-af5a-b2be85bb75bb", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
39772025-04-17T00:33:18.857ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch = Switch0
39782025-04-17T00:33:18.857ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
39792025-04-17T00:33:18.857ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a89893-516d-46ee-89ac-b788efae8041", "content-length": "682", "date": "Thu, 17 Apr 2025 00:33:18 GMT"} })
39802025-04-17T00:33:18.857ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39812025-04-17T00:33:18.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39822025-04-17T00:33:18.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39832025-04-17T00:33:18.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39842025-04-17T00:33:18.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39852025-04-17T00:33:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39862025-04-17T00:33:18.866ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39872025-04-17T00:33:18.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39882025-04-17T00:33:18.874ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 912.273318ms iteration = 25 status = {}
39892025-04-17T00:33:19.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a74bc9aa-14b4-4865-8dd9-314a651d529b roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39902025-04-17T00:33:19.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a74bc9aa-14b4-4865-8dd9-314a651d529b resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39912025-04-17T00:33:19.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2004 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a74bc9aa-14b4-4865-8dd9-314a651d529b response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39922025-04-17T00:33:20.809ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0764f747-0afe-487e-9371-8e8eb76f3986 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39932025-04-17T00:33:20.810ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0764f747-0afe-487e-9371-8e8eb76f3986 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39942025-04-17T00:33:20.812ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2397 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 0764f747-0afe-487e-9371-8e8eb76f3986 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39952025-04-17T00:33:21.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 516 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 44243e6a-9843-4856-b69b-5f82d4639e2f response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
39962025-04-17T00:33:21.832ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b4d5b2b2-648f-45aa-b35c-a70e0136ddc3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39972025-04-17T00:33:21.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b4d5b2b2-648f-45aa-b35c-a70e0136ddc3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39982025-04-17T00:33:21.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2203 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b4d5b2b2-648f-45aa-b35c-a70e0136ddc3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
39992025-04-17T00:33:22.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 304e5712-5c42-4975-b062-5bac510f87b8 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40002025-04-17T00:33:22.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 304e5712-5c42-4975-b062-5bac510f87b8 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40012025-04-17T00:33:22.856ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 304e5712-5c42-4975-b062-5bac510f87b8 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40022025-04-17T00:33:23.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2bbd7afb-eeba-4b0d-b129-dd2d65a00db3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40032025-04-17T00:33:23.876ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2bbd7afb-eeba-4b0d-b129-dd2d65a00db3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40042025-04-17T00:33:23.878ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2423 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 2bbd7afb-eeba-4b0d-b129-dd2d65a00db3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40052025-04-17T00:33:24.899ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fb60212f-19cb-4513-b411-a1c3f310d7e3 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40062025-04-17T00:33:24.899ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fb60212f-19cb-4513-b411-a1c3f310d7e3 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40072025-04-17T00:33:24.901ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2158 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = fb60212f-19cb-4513-b411-a1c3f310d7e3 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40082025-04-17T00:33:25.920ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 38ca4c2c-e798-4012-8acb-ae40da3f57bf roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40092025-04-17T00:33:25.921ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 38ca4c2c-e798-4012-8acb-ae40da3f57bf resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40102025-04-17T00:33:25.923ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2511 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 38ca4c2c-e798-4012-8acb-ae40da3f57bf response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40112025-04-17T00:33:26.942ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6a67bea8-0e1f-4330-88e2-b3645c718f05 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40122025-04-17T00:33:26.942ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6a67bea8-0e1f-4330-88e2-b3645c718f05 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40132025-04-17T00:33:26.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2265 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 6a67bea8-0e1f-4330-88e2-b3645c718f05 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40142025-04-17T00:33:27.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
40152025-04-17T00:33:27.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
40162025-04-17T00:33:27.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40172025-04-17T00:33:27.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40182025-04-17T00:33:27.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40192025-04-17T00:33:27.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40202025-04-17T00:33:27.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40212025-04-17T00:33:27.964ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40222025-04-17T00:33:27.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40232025-04-17T00:33:27.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 29663968-4cdd-4d24-a594-ec9b01d93004 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40242025-04-17T00:33:27.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 29663968-4cdd-4d24-a594-ec9b01d93004 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40252025-04-17T00:33:27.966ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.305868ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40262025-04-17T00:33:27.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40272025-04-17T00:33:27.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40282025-04-17T00:33:27.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40292025-04-17T00:33:27.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2089 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 29663968-4cdd-4d24-a594-ec9b01d93004 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40302025-04-17T00:33:27.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40312025-04-17T00:33:27.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40322025-04-17T00:33:27.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40332025-04-17T00:33:27.973ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
40342025-04-17T00:33:27.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40352025-04-17T00:33:27.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40362025-04-17T00:33:27.982ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
40372025-04-17T00:33:27.982ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.204799ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
40382025-04-17T00:33:28.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a10f2b14-cccb-4f67-b88e-495052bd4b18 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40392025-04-17T00:33:28.987ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a10f2b14-cccb-4f67-b88e-495052bd4b18 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40402025-04-17T00:33:28.988ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2059 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = a10f2b14-cccb-4f67-b88e-495052bd4b18 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40412025-04-17T00:33:30.009ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 29560b35-7929-4399-97a2-9ceabd27e2b5 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40422025-04-17T00:33:30.009ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 29560b35-7929-4399-97a2-9ceabd27e2b5 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40432025-04-17T00:33:30.011ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2454 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 29560b35-7929-4399-97a2-9ceabd27e2b5 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40442025-04-17T00:33:31.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 786242f7-df60-4b52-a643-ddb80c1176b7 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40452025-04-17T00:33:31.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 786242f7-df60-4b52-a643-ddb80c1176b7 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40462025-04-17T00:33:31.034ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2638 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 786242f7-df60-4b52-a643-ddb80c1176b7 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40472025-04-17T00:33:31.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 1e44af7e-1147-4061-ad56-398f0d0b44e4 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
40482025-04-17T00:33:31.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 1e44af7e-1147-4061-ad56-398f0d0b44e4 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
40492025-04-17T00:33:31.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3331 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 1e44af7e-1147-4061-ad56-398f0d0b44e4 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?sort_by=id_ascending
40502025-04-17T00:33:31.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 2be85071-4677-4004-8598-106e739d9593 roles = RoleSet { roles: {} } uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
40512025-04-17T00:33:31.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 2be85071-4677-4004-8598-106e739d9593 resource = Database result = Ok(()) uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
40522025-04-17T00:33:31.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2177 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42490 req_id = 2be85071-4677-4004-8598-106e739d9593 response_code = 200 uri = /metrics/collectors/c581c159-53e3-4159-ba13-1f9ca3dc97cc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNDBiZTc3NS02OTcyLTRjN2UtYjNhMS05YThlYzg2YWJlMjQifX0%3D
40532025-04-17T00:33:31.659ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 585 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 8a89c84f-a337-40f5-92a8-508608fecf1d response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
40542025-04-17T00:33:32.053ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 13e9588f-d123-4e43-8a3e-691a85621d7d roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40552025-04-17T00:33:32.054ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 13e9588f-d123-4e43-8a3e-691a85621d7d resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40562025-04-17T00:33:32.055ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2070 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 13e9588f-d123-4e43-8a3e-691a85621d7d response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40572025-04-17T00:33:33.075ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ec471bc3-e1bf-4ea9-be27-ef2204463625 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40582025-04-17T00:33:33.075ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ec471bc3-e1bf-4ea9-be27-ef2204463625 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40592025-04-17T00:33:33.077ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2251 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = ec471bc3-e1bf-4ea9-be27-ef2204463625 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40602025-04-17T00:33:34.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d8989585-924d-4dd5-a694-64985161f225 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40612025-04-17T00:33:34.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d8989585-924d-4dd5-a694-64985161f225 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40622025-04-17T00:33:34.099ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2820 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = d8989585-924d-4dd5-a694-64985161f225 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40632025-04-17T00:33:35.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f659cbbc-b131-413f-8f70-9b47c5cd12c2 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40642025-04-17T00:33:35.120ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f659cbbc-b131-413f-8f70-9b47c5cd12c2 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40652025-04-17T00:33:35.122ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2310 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f659cbbc-b131-413f-8f70-9b47c5cd12c2 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40662025-04-17T00:33:36.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 97971a50-1626-4e39-af98-f888cc49f4ac roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40672025-04-17T00:33:36.141ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 97971a50-1626-4e39-af98-f888cc49f4ac resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40682025-04-17T00:33:36.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 97971a50-1626-4e39-af98-f888cc49f4ac response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40692025-04-17T00:33:37.162ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f78924ce-7709-4e77-a1c9-60b58b7a4739 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40702025-04-17T00:33:37.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f78924ce-7709-4e77-a1c9-60b58b7a4739 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40712025-04-17T00:33:37.165ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2401 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = f78924ce-7709-4e77-a1c9-60b58b7a4739 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40722025-04-17T00:33:37.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
40732025-04-17T00:33:37.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
40742025-04-17T00:33:37.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40752025-04-17T00:33:37.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40762025-04-17T00:33:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40772025-04-17T00:33:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40782025-04-17T00:33:37.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40792025-04-17T00:33:37.963ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40802025-04-17T00:33:37.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40812025-04-17T00:33:37.964ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.387275ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40822025-04-17T00:33:37.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40832025-04-17T00:33:37.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40842025-04-17T00:33:37.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40852025-04-17T00:33:37.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40862025-04-17T00:33:37.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40872025-04-17T00:33:37.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40882025-04-17T00:33:37.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
40892025-04-17T00:33:37.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40902025-04-17T00:33:37.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40912025-04-17T00:33:37.979ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
40922025-04-17T00:33:37.979ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.487636ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
40932025-04-17T00:33:38.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 907d079f-4bb7-4b0d-af16-fd32504d610b roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40942025-04-17T00:33:38.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 907d079f-4bb7-4b0d-af16-fd32504d610b resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40952025-04-17T00:33:38.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2241 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 907d079f-4bb7-4b0d-af16-fd32504d610b response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40962025-04-17T00:33:39.205ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1be3d685-5fa7-4c97-9f0a-e22bb0be1132 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40972025-04-17T00:33:39.205ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1be3d685-5fa7-4c97-9f0a-e22bb0be1132 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40982025-04-17T00:33:39.207ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2068 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 1be3d685-5fa7-4c97-9f0a-e22bb0be1132 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
40992025-04-17T00:33:40.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 65292539-b063-4362-a304-552ebd002607 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41002025-04-17T00:33:40.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 65292539-b063-4362-a304-552ebd002607 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41012025-04-17T00:33:40.229ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2263 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 65292539-b063-4362-a304-552ebd002607 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41022025-04-17T00:33:41.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7d3af287-349e-490a-a245-c15c539ead4c roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41032025-04-17T00:33:41.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7d3af287-349e-490a-a245-c15c539ead4c resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41042025-04-17T00:33:41.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2140 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 7d3af287-349e-490a-a245-c15c539ead4c response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41052025-04-17T00:33:41.658ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 956 local_addr = [fd00:1122:3344:101::a]:39242 method = GET remote_addr = [fd00:1122:3344:101::d]:47693 req_id = 6f904265-ddd1-4e47-8685-d78ffd02c4d4 response_code = 200 uri = /ef9eec85-7cbe-4a5f-89ba-f11462071c47
41062025-04-17T00:33:42.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e8a87c82-13d9-41d0-ad39-038ec3665b4e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41072025-04-17T00:33:42.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e8a87c82-13d9-41d0-ad39-038ec3665b4e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41082025-04-17T00:33:42.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2559 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = e8a87c82-13d9-41d0-ad39-038ec3665b4e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41092025-04-17T00:33:43.291ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c7a302b3-7861-4f19-a0c4-fc2be839f853 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41102025-04-17T00:33:43.291ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c7a302b3-7861-4f19-a0c4-fc2be839f853 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41112025-04-17T00:33:43.293ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2196 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = c7a302b3-7861-4f19-a0c4-fc2be839f853 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41122025-04-17T00:33:44.312ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 82788b83-3a66-4a11-8b72-683d42d6980e roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41132025-04-17T00:33:44.312ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 82788b83-3a66-4a11-8b72-683d42d6980e resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41142025-04-17T00:33:44.314ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2547 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 82788b83-3a66-4a11-8b72-683d42d6980e response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41152025-04-17T00:33:45.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 583199b1-290f-46b8-b5f6-4d0e9aa5ca01 roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41162025-04-17T00:33:45.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 583199b1-290f-46b8-b5f6-4d0e9aa5ca01 resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41172025-04-17T00:33:45.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2464 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 583199b1-290f-46b8-b5f6-4d0e9aa5ca01 response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41182025-04-17T00:33:46.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b6b1ade0-5d0c-4b47-93e7-c9df6bce22bb roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41192025-04-17T00:33:46.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b6b1ade0-5d0c-4b47-93e7-c9df6bce22bb resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41202025-04-17T00:33:46.358ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2192 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = b6b1ade0-5d0c-4b47-93e7-c9df6bce22bb response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41212025-04-17T00:33:47.378ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 06446023-59fb-4294-a2bc-c0ac2fa9504a roles = RoleSet { roles: {} } uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41222025-04-17T00:33:47.379ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 06446023-59fb-4294-a2bc-c0ac2fa9504a resource = Database result = Ok(()) uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41232025-04-17T00:33:47.381ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2766 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33501 req_id = 06446023-59fb-4294-a2bc-c0ac2fa9504a response_code = 200 uri = /probes/ca62f5ab-0935-4588-a5c0-db1db0e32680
41242025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
41252025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
41262025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
41272025-04-17T00:33:47.961ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
41282025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
41292025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41302025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41312025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
41322025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
41332025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
41342025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41352025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41362025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
41372025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41382025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
41392025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
41402025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41412025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41422025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
41432025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
41442025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
41452025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41462025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
41472025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
41482025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41492025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41502025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
41512025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
41522025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
41532025-04-17T00:33:47.961ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41542025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
41552025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
41562025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
41572025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
41582025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
41592025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41602025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
41612025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41622025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
41632025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
41642025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41652025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41662025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41672025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41682025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
41692025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41702025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41712025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41722025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
41732025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41742025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41752025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41762025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41772025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
41782025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41792025-04-17T00:33:47.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41802025-04-17T00:33:47.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41812025-04-17T00:33:47.968ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1953, tv_nsec: 639168763 } })
41822025-04-17T00:33:47.969ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1118
41832025-04-17T00:33:47.970ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41842025-04-17T00:33:47.970ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
41852025-04-17T00:33:47.970ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41862025-04-17T00:33:47.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41872025-04-17T00:33:47.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41882025-04-17T00:33:47.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41892025-04-17T00:33:47.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41902025-04-17T00:33:47.976ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
41912025-04-17T00:33:47.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41922025-04-17T00:33:47.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41932025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
41942025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41952025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41962025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41972025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41982025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41992025-04-17T00:33:47.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42002025-04-17T00:33:47.980ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1953, tv_nsec: 651168895 } })
42012025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42022025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42032025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42042025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42052025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42062025-04-17T00:33:47.981ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1118
42072025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42082025-04-17T00:33:47.981ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42092025-04-17T00:33:47.982ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 20.860706ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
42102025-04-17T00:33:47.983ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42112025-04-17T00:33:47.983ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42122025-04-17T00:33:47.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
42132025-04-17T00:33:47.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42142025-04-17T00:33:47.985ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42152025-04-17T00:33:47.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42162025-04-17T00:33:47.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
42172025-04-17T00:33:47.987ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
42182025-04-17T00:33:47.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
42192025-04-17T00:33:47.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
42202025-04-17T00:33:47.987ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42212025-04-17T00:33:47.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42222025-04-17T00:33:47.989ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42232025-04-17T00:33:47.989ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
42242025-04-17T00:33:47.989ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42252025-04-17T00:33:47.989ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-ca62f5ab-0935-4588-a5c0-db1db0e32680.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1953, tv_nsec: 659910475 } })
42262025-04-17T00:33:47.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42272025-04-17T00:33:47.990ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1118 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
42282025-04-17T00:33:47.992ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42292025-04-17T00:33:47.992ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42302025-04-17T00:33:47.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42312025-04-17T00:33:47.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42322025-04-17T00:33:47.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42332025-04-17T00:33:47.994ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42342025-04-17T00:33:47.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42352025-04-17T00:33:47.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42362025-04-17T00:33:47.996ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 34.521959ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
42372025-04-17T00:33:47.998ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42382025-04-17T00:33:47.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42392025-04-17T00:33:47.999ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42402025-04-17T00:33:47.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42412025-04-17T00:33:47.999ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42422025-04-17T00:33:48.001ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42432025-04-17T00:33:48.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42442025-04-17T00:33:48.002ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42452025-04-17T00:33:48.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42462025-04-17T00:33:48.002ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
42472025-04-17T00:33:48.002ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 40.697955ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
42482025-04-17T00:33:48.004ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42492025-04-17T00:33:48.004ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42502025-04-17T00:33:48.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42512025-04-17T00:33:48.015ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 53.927448ms iteration = 29 status = {"physical_disks_added":0}
42522025-04-17T00:33:48.021ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 60.004481ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
42532025-04-17T00:33:48.034ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42542025-04-17T00:33:48.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42552025-04-17T00:33:48.037ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42562025-04-17T00:33:48.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42572025-04-17T00:33:48.038ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
42582025-04-17T00:33:48.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
42592025-04-17T00:33:48.039ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.062367ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42602025-04-17T00:33:48.039ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
42612025-04-17T00:33:48.039ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 77.972114ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
42622025-04-17T00:33:48.040ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.015141ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
42632025-04-17T00:33:48.041ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42642025-04-17T00:33:48.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42652025-04-17T00:33:48.041ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
42662025-04-17T00:33:48.041ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42672025-04-17T00:33:48.041ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 80.134169ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
42682025-04-17T00:33:48.042ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dda7f5de-d0b5-4ee4-900a-f42a1d54fb9d", "content-length": "26", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
42692025-04-17T00:33:48.042ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42702025-04-17T00:33:48.043ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42712025-04-17T00:33:48.046ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42722025-04-17T00:33:48.048ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 86.930709ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
42732025-04-17T00:33:48.049ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42742025-04-17T00:33:48.049ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
42752025-04-17T00:33:48.049ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 87.3576ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
42762025-04-17T00:33:48.049ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42772025-04-17T00:33:48.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42782025-04-17T00:33:48.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ca62f5ab-0935-4588-a5c0-db1db0e32680, lookup_type: ById(ca62f5ab-0935-4588-a5c0-db1db0e32680) } result = Ok(())
42792025-04-17T00:33:48.051ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42802025-04-17T00:33:48.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42812025-04-17T00:33:48.053ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 91.156276ms iteration = 26 status = 0
42822025-04-17T00:33:48.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42832025-04-17T00:33:48.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4586b28a-678c-4f28-94b3-ec2bd89a63c3, lookup_type: ById(4586b28a-678c-4f28-94b3-ec2bd89a63c3) } result = Ok(())
42842025-04-17T00:33:48.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42852025-04-17T00:33:48.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42862025-04-17T00:33:48.070ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1126 zone_address = fd00:1122:3344:101::2
42872025-04-17T00:33:48.070ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42882025-04-17T00:33:48.070ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7fa0cd9-3e58-4c05-b515-2745a70e9399", "content-length": "26", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
42892025-04-17T00:33:48.070ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1129 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42902025-04-17T00:33:48.071ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1126 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 zone_address = fd00:1122:3344:101::2
42912025-04-17T00:33:48.071ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42922025-04-17T00:33:48.071ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48caa051-3aa9-4055-876d-54c6375998dc", "content-length": "26", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
42932025-04-17T00:33:48.071ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1129 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42942025-04-17T00:33:48.072ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42952025-04-17T00:33:48.072ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1163 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
42962025-04-17T00:33:48.075ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 original_time_created = 2025-04-17 00:21:13.964362 UTC target_id = 4586b28a-678c-4f28-94b3-ec2bd89a63c3 time_created = 2025-04-17 00:21:13.964362 UTC
42972025-04-17T00:33:48.075ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 113.288814ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4586b28a-678c-4f28-94b3-ec2bd89a63c3","time_created":"2025-04-17T00:21:13.964362Z"}
42982025-04-17T00:33:48.085ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 123.429269ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
42992025-04-17T00:33:48.120ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43002025-04-17T00:33:48.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43012025-04-17T00:33:48.125ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 164.101433ms iteration = 26 status = {}
43022025-04-17T00:33:48.128ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43032025-04-17T00:33:48.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43042025-04-17T00:33:48.130ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43052025-04-17T00:33:48.133ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43062025-04-17T00:33:48.133ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43072025-04-17T00:33:48.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43082025-04-17T00:33:48.134ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43092025-04-17T00:33:48.136ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43102025-04-17T00:33:48.136ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43112025-04-17T00:33:48.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43122025-04-17T00:33:48.155ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43132025-04-17T00:33:48.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43142025-04-17T00:33:48.172ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
43152025-04-17T00:33:48.172ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
43162025-04-17T00:33:48.173ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cfe399e-e9db-4f0c-800b-91e2d5e3447d", "content-length": "372", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
43172025-04-17T00:33:48.173ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43182025-04-17T00:33:48.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43192025-04-17T00:33:48.174ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 213.081307ms iteration = 26 status = null
43202025-04-17T00:33:48.229ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
43212025-04-17T00:33:48.229ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
43222025-04-17T00:33:48.230ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f863f7c1-fad9-45e0-9379-7fa73d9061d3", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
43232025-04-17T00:33:48.230ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43242025-04-17T00:33:48.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43252025-04-17T00:33:48.232ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
43262025-04-17T00:33:48.232ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d43743a-fcab-4185-8a76-304049ceae8a", "content-length": "691", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
43272025-04-17T00:33:48.232ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
43282025-04-17T00:33:48.232ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
43292025-04-17T00:33:48.232ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
43302025-04-17T00:33:48.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43312025-04-17T00:33:48.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43322025-04-17T00:33:48.234ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 273.377056ms iteration = 27 status = {}
43332025-04-17T00:33:48.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43342025-04-17T00:33:48.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43352025-04-17T00:33:48.239ZINFOnexus (BackgroundTasks): VPC route manager sled ca62f5ab-0935-4588-a5c0-db1db0e32680 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
43362025-04-17T00:33:48.239ZDEBGnexus (BackgroundTasks): client request SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
43372025-04-17T00:33:48.239ZDEBGnexus (BackgroundTasks): client response SledAgent = ca62f5ab-0935-4588-a5c0-db1db0e32680 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2e2f1011-0e33-4e34-9e01-20b272b646e4", "date": "Thu, 17 Apr 2025 00:33:48 GMT"} })
43382025-04-17T00:33:48.241ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 280.303611ms iteration = 27 status = {}
43392025-04-17T00:33:48.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43402025-04-17T00:33:48.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43412025-04-17T00:33:48.273ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1cd2ca0f-44cf-49b6-8dce-d9be01c0efef, rack_id: f4db4a97-be6c-41b1-96e9-109f9dbfc809, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(088b9026-f1da-4064-a2bf-cd02c431c3c5) } rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
43422025-04-17T00:33:48.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43432025-04-17T00:33:48.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43442025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
43452025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 088b9026-f1da-4064-a2bf-cd02c431c3c5
43462025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
43472025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
43482025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43492025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43502025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43512025-04-17T00:33:48.286ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
43522025-04-17T00:33:48.287ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43532025-04-17T00:33:48.287ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
43542025-04-17T00:33:48.287ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809
43552025-04-17T00:33:48.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43562025-04-17T00:33:48.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43572025-04-17T00:33:48.289ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 switch_location = Switch0
43582025-04-17T00:33:48.289ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f4db4a97-be6c-41b1-96e9-109f9dbfc809 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports