Max level shown:
12025-06-04T01:30:02.841ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
22025-06-04T01:30:02.842ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32025-06-04T01:30:02.843ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
42025-06-04T01:30:02.844ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
52025-06-04T01:30:02.846ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
62025-06-04T01:30:02.847ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
72025-06-04T01:30:02.847ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
82025-06-04T01:30:02.847ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
92025-06-04T01:30:02.849ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
102025-06-04T01:30:02.849ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
112025-06-04T01:30:02.851ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
122025-06-04T01:30:02.851ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
132025-06-04T01:30:02.852ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
142025-06-04T01:30:02.853ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
152025-06-04T01:30:02.854ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
162025-06-04T01:30:02.856ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
172025-06-04T01:30:02.856ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
182025-06-04T01:30:02.856ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
192025-06-04T01:30:02.858ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
202025-06-04T01:30:02.859ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
212025-06-04T01:30:02.859ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
222025-06-04T01:30:02.859ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
232025-06-04T01:30:02.863ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
242025-06-04T01:30:02.863ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
252025-06-04T01:30:02.865ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
262025-06-04T01:30:02.865ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
272025-06-04T01:30:02.867ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
282025-06-04T01:30:02.867ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
292025-06-04T01:30:02.869ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
302025-06-04T01:30:02.869ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
312025-06-04T01:30:02.870ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
322025-06-04T01:30:02.871ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
332025-06-04T01:30:02.871ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
342025-06-04T01:30:02.871ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
352025-06-04T01:30:02.873ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
362025-06-04T01:30:02.873ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
372025-06-04T01:30:02.874ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
382025-06-04T01:30:02.874ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
392025-06-04T01:30:02.875ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
402025-06-04T01:30:02.877ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
412025-06-04T01:30:02.877ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
422025-06-04T01:30:02.877ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
432025-06-04T01:30:02.878ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
442025-06-04T01:30:02.878ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
452025-06-04T01:30:02.880ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
462025-06-04T01:30:02.881ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
472025-06-04T01:30:02.882ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
482025-06-04T01:30:02.882ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
492025-06-04T01:30:02.885ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
502025-06-04T01:30:02.885ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
512025-06-04T01:30:02.887ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
522025-06-04T01:30:02.887ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
532025-06-04T01:30:02.889ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
542025-06-04T01:30:02.890ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
552025-06-04T01:30:02.892ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
562025-06-04T01:30:02.895ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
572025-06-04T01:30:02.895ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
582025-06-04T01:30:02.895ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
592025-06-04T01:30:02.898ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
602025-06-04T01:30:02.898ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
612025-06-04T01:30:02.900ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
622025-06-04T01:30:02.901ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
632025-06-04T01:30:02.902ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
642025-06-04T01:30:02.902ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
652025-06-04T01:30:02.905ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
662025-06-04T01:30:02.908ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
672025-06-04T01:30:02.908ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
682025-06-04T01:30:02.908ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
692025-06-04T01:30:02.911ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 69800 local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8ff6c94d-6775-43df-bbb1-2d7ae90bb61f response_code = 200 uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
702025-06-04T01:30:02.912ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
712025-06-04T01:30:02.912ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
722025-06-04T01:30:02.914ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
732025-06-04T01:30:02.914ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
742025-06-04T01:30:02.916ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
752025-06-04T01:30:02.917ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
762025-06-04T01:30:02.917ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
772025-06-04T01:30:02.917ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
782025-06-04T01:30:02.918ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
792025-06-04T01:30:02.918ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
802025-06-04T01:30:02.920ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
812025-06-04T01:30:02.920ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
822025-06-04T01:30:02.921ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
832025-06-04T01:30:02.921ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
842025-06-04T01:30:02.923ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
852025-06-04T01:30:02.924ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
862025-06-04T01:30:02.924ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
872025-06-04T01:30:02.924ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
882025-06-04T01:30:02.925ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
892025-06-04T01:30:02.926ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
902025-06-04T01:30:02.926ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
912025-06-04T01:30:02.927ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
922025-06-04T01:30:02.930ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
932025-06-04T01:30:02.930ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
942025-06-04T01:30:02.932ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
952025-06-04T01:30:02.932ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
962025-06-04T01:30:02.933ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
972025-06-04T01:30:02.933ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
982025-06-04T01:30:02.934ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
992025-06-04T01:30:02.935ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1002025-06-04T01:30:02.936ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1012025-06-04T01:30:02.937ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
1022025-06-04T01:30:02.937ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1032025-06-04T01:30:02.937ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1042025-06-04T01:30:02.939ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1052025-06-04T01:30:02.939ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1062025-06-04T01:30:02.945ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1072025-06-04T01:30:02.946ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1082025-06-04T01:30:02.947ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1092025-06-04T01:30:02.949ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
1102025-06-04T01:30:02.949ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1112025-06-04T01:30:02.949ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1122025-06-04T01:30:02.950ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1132025-06-04T01:30:02.950ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1142025-06-04T01:30:02.952ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1152025-06-04T01:30:02.952ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1162025-06-04T01:30:02.955ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1172025-06-04T01:30:02.955ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1182025-06-04T01:30:02.958ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1192025-06-04T01:30:02.958ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1202025-06-04T01:30:02.960ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1212025-06-04T01:30:02.960ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1222025-06-04T01:30:02.963ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1232025-06-04T01:30:02.963ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1242025-06-04T01:30:02.966ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1252025-06-04T01:30:02.969ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1262025-06-04T01:30:02.969ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1272025-06-04T01:30:02.969ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1282025-06-04T01:30:02.971ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1292025-06-04T01:30:02.971ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1302025-06-04T01:30:02.973ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1312025-06-04T01:30:02.973ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1322025-06-04T01:30:02.976ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1332025-06-04T01:30:02.976ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1342025-06-04T01:30:02.978ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1352025-06-04T01:30:02.981ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1362025-06-04T01:30:02.981ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1372025-06-04T01:30:02.981ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1382025-06-04T01:30:03.018ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 105751 local_addr = 172.30.2.7:443 method = GET remote_addr = 10.151.2.100:45663 req_id = 8c8afc0d-6828-4405-a2fb-77047ae3a256 response_code = 200 uri = /v1/instances/instance-506f220f14c/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-25de54cd75b
1392025-06-04T01:30:03.511ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1402025-06-04T01:30:03.511ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1412025-06-04T01:30:03.514ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1422025-06-04T01:30:03.514ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1432025-06-04T01:30:03.516ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1442025-06-04T01:30:03.517ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
1452025-06-04T01:30:03.517ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1462025-06-04T01:30:03.517ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1472025-06-04T01:30:03.519ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1482025-06-04T01:30:03.519ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1492025-06-04T01:30:03.520ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1502025-06-04T01:30:03.520ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1512025-06-04T01:30:03.522ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1522025-06-04T01:30:03.522ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1532025-06-04T01:30:03.523ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1542025-06-04T01:30:03.524ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
1552025-06-04T01:30:03.524ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1562025-06-04T01:30:03.524ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1572025-06-04T01:30:03.526ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1582025-06-04T01:30:03.527ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
1592025-06-04T01:30:03.527ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1602025-06-04T01:30:03.527ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1612025-06-04T01:30:03.531ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1622025-06-04T01:30:03.531ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1632025-06-04T01:30:03.533ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1642025-06-04T01:30:03.533ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1652025-06-04T01:30:03.534ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1662025-06-04T01:30:03.534ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1672025-06-04T01:30:03.536ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1682025-06-04T01:30:03.536ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1692025-06-04T01:30:03.537ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1702025-06-04T01:30:03.538ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
1712025-06-04T01:30:03.538ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1722025-06-04T01:30:03.538ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1732025-06-04T01:30:03.540ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1742025-06-04T01:30:03.540ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1752025-06-04T01:30:03.541ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
1762025-06-04T01:30:03.542ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
1772025-06-04T01:30:03.543ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
1782025-06-04T01:30:03.544ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
1792025-06-04T01:30:03.544ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1802025-06-04T01:30:03.544ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1812025-06-04T01:30:03.545ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1822025-06-04T01:30:03.546ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1832025-06-04T01:30:03.547ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1842025-06-04T01:30:03.548ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1852025-06-04T01:30:03.550ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1862025-06-04T01:30:03.551ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1872025-06-04T01:30:03.553ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1882025-06-04T01:30:03.553ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1892025-06-04T01:30:03.555ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1902025-06-04T01:30:03.555ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1912025-06-04T01:30:03.557ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1922025-06-04T01:30:03.557ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1932025-06-04T01:30:03.560ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1942025-06-04T01:30:03.563ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1952025-06-04T01:30:03.563ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1962025-06-04T01:30:03.563ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1972025-06-04T01:30:03.565ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1982025-06-04T01:30:03.565ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
1992025-06-04T01:30:03.567ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2002025-06-04T01:30:03.568ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2012025-06-04T01:30:03.570ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2022025-06-04T01:30:03.570ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2032025-06-04T01:30:03.572ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2042025-06-04T01:30:03.575ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2052025-06-04T01:30:03.575ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2062025-06-04T01:30:03.575ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2072025-06-04T01:30:03.578ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2082025-06-04T01:30:03.578ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2092025-06-04T01:30:03.581ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2102025-06-04T01:30:03.581ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2112025-06-04T01:30:03.583ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2122025-06-04T01:30:03.583ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2132025-06-04T01:30:03.585ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2142025-06-04T01:30:03.585ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2152025-06-04T01:30:03.587ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2162025-06-04T01:30:03.589ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2172025-06-04T01:30:03.633ZINFOnexus (dropshot_external): set intended instance state actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true changed = true file = nexus/db-queries/src/db/datastore/instance.rs:812 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d intended_state = stopped local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2182025-06-04T01:30:03.663ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
2192025-06-04T01:30:03.663ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
2202025-06-04T01:30:03.665ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
2212025-06-04T01:30:03.665ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
2222025-06-04T01:30:03.667ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
2232025-06-04T01:30:03.669ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
2242025-06-04T01:30:03.669ZDEBGnexus: client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e/state
2252025-06-04T01:30:03.669ZDEBGnexus: client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0311dc8-3546-4071-bc3c-6bb067b47901", "content-length": "151", "date": "Wed, 04 Jun 2025 01:30:03 GMT"} })
2262025-06-04T01:30:03.669ZINFOnexus (dropshot_external): received new VMM runtime state from sled agent actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true file = nexus/src/app/instance.rs:2140 local_addr = 172.30.2.7:443 method = POST migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = a8435d87-6398-4e8f-9fda-269bcd96ac6e remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2025-06-04T01:30:03.669676704Z }
2272025-06-04T01:30:03.669ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2282025-06-04T01:30:03.670ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2292025-06-04T01:30:03.710ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 20 transaction = vmm_and_migration_update_runtime
2302025-06-04T01:30:03.736ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = vmm_and_migration_update_runtime
2312025-06-04T01:30:03.738ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2322025-06-04T01:30:03.738ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2332025-06-04T01:30:03.740ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2342025-06-04T01:30:03.741ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2352025-06-04T01:30:03.744ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2362025-06-04T01:30:03.744ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2372025-06-04T01:30:03.746ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2382025-06-04T01:30:03.746ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2392025-06-04T01:30:03.749ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2402025-06-04T01:30:03.752ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2412025-06-04T01:30:03.752ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2422025-06-04T01:30:03.752ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2432025-06-04T01:30:03.755ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 246044 local_addr = 172.30.2.7:443 method = POST remote_addr = 10.151.2.100:45663 req_id = 88512288-e95a-40da-b450-148629e9a3c2 response_code = 202 uri = /v1/instances/instance-506f220f14c/stop?project=proj-25de54cd75b
2442025-06-04T01:30:03.756ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2452025-06-04T01:30:03.756ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2462025-06-04T01:30:03.758ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2472025-06-04T01:30:03.758ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2482025-06-04T01:30:03.760ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
2492025-06-04T01:30:03.761ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
2502025-06-04T01:30:03.761ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2512025-06-04T01:30:03.761ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2522025-06-04T01:30:03.763ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2532025-06-04T01:30:03.763ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2542025-06-04T01:30:03.764ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2552025-06-04T01:30:03.764ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2562025-06-04T01:30:03.766ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2572025-06-04T01:30:03.766ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2582025-06-04T01:30:03.767ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
2592025-06-04T01:30:03.768ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
2602025-06-04T01:30:03.768ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2612025-06-04T01:30:03.768ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2622025-06-04T01:30:03.769ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
2632025-06-04T01:30:03.771ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
2642025-06-04T01:30:03.771ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2652025-06-04T01:30:03.771ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2662025-06-04T01:30:03.775ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2672025-06-04T01:30:03.775ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2682025-06-04T01:30:03.776ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2692025-06-04T01:30:03.776ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2702025-06-04T01:30:03.778ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2712025-06-04T01:30:03.778ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2722025-06-04T01:30:03.779ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2732025-06-04T01:30:03.779ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2742025-06-04T01:30:03.781ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
2752025-06-04T01:30:03.782ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
2762025-06-04T01:30:03.782ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2772025-06-04T01:30:03.782ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2782025-06-04T01:30:03.784ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2792025-06-04T01:30:03.784ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2802025-06-04T01:30:03.785ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
2812025-06-04T01:30:03.785ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
2822025-06-04T01:30:03.787ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
2832025-06-04T01:30:03.788ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
2842025-06-04T01:30:03.788ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2852025-06-04T01:30:03.788ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2862025-06-04T01:30:03.789ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2872025-06-04T01:30:03.789ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2882025-06-04T01:30:03.791ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2892025-06-04T01:30:03.791ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2902025-06-04T01:30:03.793ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2912025-06-04T01:30:03.793ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2922025-06-04T01:30:03.795ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2932025-06-04T01:30:03.795ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2942025-06-04T01:30:03.798ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2952025-06-04T01:30:03.798ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2962025-06-04T01:30:03.800ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2972025-06-04T01:30:03.800ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2982025-06-04T01:30:03.803ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
2992025-06-04T01:30:03.805ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3002025-06-04T01:30:03.805ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3012025-06-04T01:30:03.805ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3022025-06-04T01:30:03.808ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3032025-06-04T01:30:03.808ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3042025-06-04T01:30:03.811ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3052025-06-04T01:30:03.811ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3062025-06-04T01:30:03.813ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3072025-06-04T01:30:03.813ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3082025-06-04T01:30:03.815ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3092025-06-04T01:30:03.817ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3102025-06-04T01:30:03.825ZINFOnexus (dropshot_external): set intended instance state actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true changed = true file = nexus/db-queries/src/db/datastore/instance.rs:812 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d intended_state = destroyed local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3112025-06-04T01:30:03.825ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3122025-06-04T01:30:03.825ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3132025-06-04T01:30:03.826ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
3142025-06-04T01:30:03.827ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
3152025-06-04T01:30:03.827ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3162025-06-04T01:30:03.828ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3172025-06-04T01:30:03.832ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete
3182025-06-04T01:30:03.832ZINFOnexus (SEC): saga create dag = {"end_node":5,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,0,null],[3,5,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_delete.instance_delete_nat","label":"InstanceDeleteNat","name":"no_result4"}},{"Action":{"action_name":"instance_delete.instance_delete_record","label":"InstanceDeleteRecord","name":"no_result1"}},{"Action":{"action_name":"instance_delete.delete_network_interfaces","label":"DeleteNetworkInterfaces","name":"no_result2"}},{"Action":{"action_name":"instance_delete.deallocate_external_ip","label":"DeallocateExternalIp","name":"no_result3"}},{"Start":{"params":{"authz_instance":{"key":"5abad387-28b5-4eac-817c-81deb58a742d","lookup_type":{"ByName":"instance-506f220f14c"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ByName":"proj-25de54cd75b"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}},"boundary_switches":["switch0"],"instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":"c63ca8c6-5bef-4746-bdc3-504c0087eb2b","hostname":"localshark","identity":{"description":"","id":"5abad387-28b5-4eac-817c-81deb58a742d","name":"instance-506f220f14c","time_created":"2025-06-04T01:29:43.103614Z","time_deleted":null,"time_modified":"2025-06-04T01:30:03.589873Z"},"intended_state":"Stopped","memory":1073741824,"ncpus":2,"project_id":"6a575fb0-c195-4d2a-ac26-16b83f49e832","runtime_state":{"dst_propolis_id":null,"gen":3,"migration_id":null,"nexus_state":"Vmm","propolis_id":"a8435d87-6398-4e8f-9fda-269bcd96ac6e","time_last_auto_restarted":null,"time_updated":"2025-06-04T01:29:43.103614Z"},"updater_gen":1,"updater_id":null,"user_data":[]},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","silo_user_id":"aa424c47-6907-47fb-925a-807ae77df887"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"instance-delete","start_node":4} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3192025-06-04T01:30:03.832ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete
3202025-06-04T01:30:03.872ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete
3212025-06-04T01:30:03.872ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3222025-06-04T01:30:03.872ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3232025-06-04T01:30:03.872ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3242025-06-04T01:30:03.883ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3252025-06-04T01:30:03.883ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3262025-06-04T01:30:03.887ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3272025-06-04T01:30:03.887ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3282025-06-04T01:30:03.891ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3292025-06-04T01:30:03.891ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3302025-06-04T01:30:03.893ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3312025-06-04T01:30:03.893ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3322025-06-04T01:30:03.895ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3332025-06-04T01:30:03.895ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3342025-06-04T01:30:03.896ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3352025-06-04T01:30:03.896ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3362025-06-04T01:30:03.899ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3372025-06-04T01:30:03.899ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3382025-06-04T01:30:03.901ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3392025-06-04T01:30:03.902ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3402025-06-04T01:30:03.904ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3412025-06-04T01:30:03.905ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3422025-06-04T01:30:03.907ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3432025-06-04T01:30:03.910ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3442025-06-04T01:30:03.910ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d
3452025-06-04T01:30:03.910ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3462025-06-04T01:30:03.910ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3472025-06-04T01:30:03.912ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3482025-06-04T01:30:03.912ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3492025-06-04T01:30:03.920ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3502025-06-04T01:30:03.920ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteNat
3512025-06-04T01:30:03.935ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3522025-06-04T01:30:03.935ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3532025-06-04T01:30:03.937ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
3542025-06-04T01:30:03.938ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
3552025-06-04T01:30:03.938ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3562025-06-04T01:30:03.938ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3572025-06-04T01:30:03.942ZDEBGnexus: notifying dendrite of updates instance_id = Some(5abad387-28b5-4eac-817c-81deb58a742d (instance)) switch = switch0
3582025-06-04T01:30:03.942ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1741, tv_nsec: 35648734 } })
3592025-06-04T01:30:03.943ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1179
3602025-06-04T01:30:03.972ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
3612025-06-04T01:30:03.972ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3622025-06-04T01:30:03.973ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8825e10c-5dbb-4d92-be99-4fa1d5d62815", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:03 GMT"} })
3632025-06-04T01:30:03.973ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3642025-06-04T01:30:03.973ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3652025-06-04T01:30:04.003ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3662025-06-04T01:30:04.003ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3672025-06-04T01:30:04.008ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3682025-06-04T01:30:04.008ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3692025-06-04T01:30:04.011ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3702025-06-04T01:30:04.012ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3712025-06-04T01:30:04.014ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3722025-06-04T01:30:04.014ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3732025-06-04T01:30:04.017ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3742025-06-04T01:30:04.017ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3752025-06-04T01:30:04.020ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3762025-06-04T01:30:04.020ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3772025-06-04T01:30:04.022ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3782025-06-04T01:30:04.024ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3792025-06-04T01:30:04.024ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3802025-06-04T01:30:04.025ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_node = InstanceDeleteRecord
3812025-06-04T01:30:04.068ZDEBGnexus (SEC): saga log event new_state = N001 failed sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3822025-06-04T01:30:04.068ZDEBGnexus (SecStore): recording saga event event_type = Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }) node_id = 1 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3832025-06-04T01:30:04.071ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Unwinding saga_id = 22b9504d-80f9-4776-995d-a711816c264f sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3842025-06-04T01:30:04.072ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = unwinding saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3852025-06-04T01:30:04.115ZDEBGnexus (SEC): saga log event new_state = N000 undo_started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3862025-06-04T01:30:04.115ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 0 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3872025-06-04T01:30:04.119ZDEBGnexus (SEC): saga log event new_state = N000 undo_finished sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3882025-06-04T01:30:04.119ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 0 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3892025-06-04T01:30:04.122ZDEBGnexus (SEC): saga log event new_state = N004 undo_started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3902025-06-04T01:30:04.122ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 4 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3912025-06-04T01:30:04.126ZDEBGnexus (SEC): saga log event new_state = N004 undo_finished sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3922025-06-04T01:30:04.126ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 4 saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3932025-06-04T01:30:04.129ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 22b9504d-80f9-4776-995d-a711816c264f sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
3942025-06-04T01:30:04.129ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 22b9504d-80f9-4776-995d-a711816c264f
3952025-06-04T01:30:04.135ZWARNnexus (SEC): saga finished action_error_node_name = "no_result1" action_error_source = ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} } file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1044 result = failure saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac undo_result = success
3962025-06-04T01:30:04.135ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 22b9504d-80f9-4776-995d-a711816c264f saga_name = instance-delete saga_result = SagaResult { saga_id: 22b9504d-80f9-4776-995d-a711816c264f, saga_log: SagaLog { saga_id: 22b9504d-80f9-4776-995d-a711816c264f, unwinding: true, events: [N004 started, N004 succeeded, N000 started, N000 succeeded, N001 started, N001 failed, N000 undo_started, N000 undo_finished, N004 undo_started, N004 undo_finished], node_status: {0: UndoFinished, 1: Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }), 4: UndoFinished} }, kind: Err(SagaResultErr { error_node_name: "no_result1", error_source: ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }, undo_failure: None }) }
3972025-06-04T01:30:04.135ZINFOnexus (dropshot_external): request completed error_message_external = cannot delete instance: instance is running or has not yet fully stopped error_message_internal = cannot delete instance: instance is running or has not yet fully stopped (with internal context: saga ACTION error at node "no_result1") file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:855 latency_us = 379542 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = b166e658-3a97-45a0-8e87-958e3ca43cd0 response_code = 400 uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
3982025-06-04T01:30:04.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = 1fec6429-bb5c-4f4e-b938-032011445ae7 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
3992025-06-04T01:30:04.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = 1fec6429-bb5c-4f4e-b938-032011445ae7 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
4002025-06-04T01:30:04.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2630 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = 1fec6429-bb5c-4f4e-b938-032011445ae7 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
4012025-06-04T01:30:04.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = 741bf207-c2fa-4b12-933f-559650a9d744 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
4022025-06-04T01:30:04.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = 741bf207-c2fa-4b12-933f-559650a9d744 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
4032025-06-04T01:30:04.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3460 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = 741bf207-c2fa-4b12-933f-559650a9d744 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
4042025-06-04T01:30:05.138ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4052025-06-04T01:30:05.138ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4062025-06-04T01:30:05.140ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4072025-06-04T01:30:05.140ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4082025-06-04T01:30:05.142ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4092025-06-04T01:30:05.143ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
4102025-06-04T01:30:05.143ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4112025-06-04T01:30:05.143ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4122025-06-04T01:30:05.145ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4132025-06-04T01:30:05.145ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4142025-06-04T01:30:05.147ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4152025-06-04T01:30:05.147ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4162025-06-04T01:30:05.149ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4172025-06-04T01:30:05.149ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4182025-06-04T01:30:05.150ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4192025-06-04T01:30:05.152ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
4202025-06-04T01:30:05.152ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4212025-06-04T01:30:05.152ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4222025-06-04T01:30:05.154ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4232025-06-04T01:30:05.155ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
4242025-06-04T01:30:05.155ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4252025-06-04T01:30:05.155ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4262025-06-04T01:30:05.159ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4272025-06-04T01:30:05.159ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4282025-06-04T01:30:05.161ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4292025-06-04T01:30:05.161ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4302025-06-04T01:30:05.163ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4312025-06-04T01:30:05.163ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4322025-06-04T01:30:05.164ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4332025-06-04T01:30:05.165ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4342025-06-04T01:30:05.166ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4352025-06-04T01:30:05.167ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
4362025-06-04T01:30:05.167ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4372025-06-04T01:30:05.167ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4382025-06-04T01:30:05.169ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4392025-06-04T01:30:05.169ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4402025-06-04T01:30:05.170ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4412025-06-04T01:30:05.170ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4422025-06-04T01:30:05.172ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4432025-06-04T01:30:05.173ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
4442025-06-04T01:30:05.173ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4452025-06-04T01:30:05.173ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4462025-06-04T01:30:05.174ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4472025-06-04T01:30:05.174ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4482025-06-04T01:30:05.176ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4492025-06-04T01:30:05.176ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4502025-06-04T01:30:05.177ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4512025-06-04T01:30:05.178ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4522025-06-04T01:30:05.180ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4532025-06-04T01:30:05.180ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4542025-06-04T01:30:05.182ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4552025-06-04T01:30:05.182ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4562025-06-04T01:30:05.185ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4572025-06-04T01:30:05.185ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4582025-06-04T01:30:05.187ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4592025-06-04T01:30:05.190ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4602025-06-04T01:30:05.190ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4612025-06-04T01:30:05.190ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4622025-06-04T01:30:05.192ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4632025-06-04T01:30:05.192ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4642025-06-04T01:30:05.195ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4652025-06-04T01:30:05.195ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4662025-06-04T01:30:05.196ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4672025-06-04T01:30:05.196ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4682025-06-04T01:30:05.199ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4692025-06-04T01:30:05.202ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4702025-06-04T01:30:05.239ZINFOnexus (dropshot_external): set intended instance state actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true changed = false file = nexus/db-queries/src/db/datastore/instance.rs:812 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d intended_state = destroyed local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
4712025-06-04T01:30:05.239ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
4722025-06-04T01:30:05.240ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
4732025-06-04T01:30:05.241ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
4742025-06-04T01:30:05.243ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
4752025-06-04T01:30:05.243ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
4762025-06-04T01:30:05.243ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
4772025-06-04T01:30:05.246ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete
4782025-06-04T01:30:05.246ZINFOnexus (SEC): saga create dag = {"end_node":5,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,0,null],[3,5,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_delete.instance_delete_nat","label":"InstanceDeleteNat","name":"no_result4"}},{"Action":{"action_name":"instance_delete.instance_delete_record","label":"InstanceDeleteRecord","name":"no_result1"}},{"Action":{"action_name":"instance_delete.delete_network_interfaces","label":"DeleteNetworkInterfaces","name":"no_result2"}},{"Action":{"action_name":"instance_delete.deallocate_external_ip","label":"DeallocateExternalIp","name":"no_result3"}},{"Start":{"params":{"authz_instance":{"key":"5abad387-28b5-4eac-817c-81deb58a742d","lookup_type":{"ByName":"instance-506f220f14c"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ByName":"proj-25de54cd75b"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}},"boundary_switches":["switch0"],"instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":"c63ca8c6-5bef-4746-bdc3-504c0087eb2b","hostname":"localshark","identity":{"description":"","id":"5abad387-28b5-4eac-817c-81deb58a742d","name":"instance-506f220f14c","time_created":"2025-06-04T01:29:43.103614Z","time_deleted":null,"time_modified":"2025-06-04T01:30:03.817850Z"},"intended_state":"Destroyed","memory":1073741824,"ncpus":2,"project_id":"6a575fb0-c195-4d2a-ac26-16b83f49e832","runtime_state":{"dst_propolis_id":null,"gen":3,"migration_id":null,"nexus_state":"Vmm","propolis_id":"a8435d87-6398-4e8f-9fda-269bcd96ac6e","time_last_auto_restarted":null,"time_updated":"2025-06-04T01:29:43.103614Z"},"updater_gen":1,"updater_id":null,"user_data":[]},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","silo_user_id":"aa424c47-6907-47fb-925a-807ae77df887"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"instance-delete","start_node":4} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
4792025-06-04T01:30:05.246ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete
4802025-06-04T01:30:05.280ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete
4812025-06-04T01:30:05.280ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
4822025-06-04T01:30:05.280ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
4832025-06-04T01:30:05.280ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 50268305-c950-409c-9416-035f25ab5792
4842025-06-04T01:30:05.284ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
4852025-06-04T01:30:05.284ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 50268305-c950-409c-9416-035f25ab5792
4862025-06-04T01:30:05.287ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
4872025-06-04T01:30:05.287ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 50268305-c950-409c-9416-035f25ab5792
4882025-06-04T01:30:05.290ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4892025-06-04T01:30:05.291ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4902025-06-04T01:30:05.293ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4912025-06-04T01:30:05.293ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4922025-06-04T01:30:05.294ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4932025-06-04T01:30:05.295ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4942025-06-04T01:30:05.296ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4952025-06-04T01:30:05.296ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4962025-06-04T01:30:05.298ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4972025-06-04T01:30:05.299ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4982025-06-04T01:30:05.301ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
4992025-06-04T01:30:05.301ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5002025-06-04T01:30:05.304ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5012025-06-04T01:30:05.304ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5022025-06-04T01:30:05.306ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5032025-06-04T01:30:05.308ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5042025-06-04T01:30:05.308ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d
5052025-06-04T01:30:05.308ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5062025-06-04T01:30:05.309ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5072025-06-04T01:30:05.311ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5082025-06-04T01:30:05.311ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5092025-06-04T01:30:05.313ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
5102025-06-04T01:30:05.313ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5112025-06-04T01:30:05.313ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteNat
5122025-06-04T01:30:05.315ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
5132025-06-04T01:30:05.315ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
5142025-06-04T01:30:05.315ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
5152025-06-04T01:30:05.317ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
5162025-06-04T01:30:05.318ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
5172025-06-04T01:30:05.318ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
5182025-06-04T01:30:05.318ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
5192025-06-04T01:30:05.323ZDEBGnexus: notifying dendrite of updates instance_id = Some(5abad387-28b5-4eac-817c-81deb58a742d (instance)) switch = switch0
5202025-06-04T01:30:05.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1742, tv_nsec: 417129932 } })
5212025-06-04T01:30:05.324ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1179
5222025-06-04T01:30:05.354ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
5232025-06-04T01:30:05.354ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5242025-06-04T01:30:05.354ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9261774-b738-430e-80ae-68d67bf5548d", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:05 GMT"} })
5252025-06-04T01:30:05.354ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5262025-06-04T01:30:05.355ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5272025-06-04T01:30:05.385ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5282025-06-04T01:30:05.385ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = 50268305-c950-409c-9416-035f25ab5792
5292025-06-04T01:30:05.389ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5302025-06-04T01:30:05.389ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 50268305-c950-409c-9416-035f25ab5792
5312025-06-04T01:30:05.392ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5322025-06-04T01:30:05.392ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5332025-06-04T01:30:05.394ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5342025-06-04T01:30:05.394ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5352025-06-04T01:30:05.397ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5362025-06-04T01:30:05.397ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5372025-06-04T01:30:05.399ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5382025-06-04T01:30:05.399ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5392025-06-04T01:30:05.401ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5402025-06-04T01:30:05.404ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5412025-06-04T01:30:05.404ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5422025-06-04T01:30:05.404ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_node = InstanceDeleteRecord
5432025-06-04T01:30:05.471ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 5461 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 719ca50b-7130-48e8-8f60-c2aca9d16629 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
5442025-06-04T01:30:05.516ZDEBGnexus (SEC): saga log event new_state = N001 failed sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5452025-06-04T01:30:05.516ZDEBGnexus (SecStore): recording saga event event_type = Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }) node_id = 1 saga_id = 50268305-c950-409c-9416-035f25ab5792
5462025-06-04T01:30:05.520ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Unwinding saga_id = 50268305-c950-409c-9416-035f25ab5792 sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5472025-06-04T01:30:05.520ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = unwinding saga_id = 50268305-c950-409c-9416-035f25ab5792
5482025-06-04T01:30:05.527ZDEBGnexus (SEC): saga log event new_state = N000 undo_started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5492025-06-04T01:30:05.527ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 0 saga_id = 50268305-c950-409c-9416-035f25ab5792
5502025-06-04T01:30:05.531ZDEBGnexus (SEC): saga log event new_state = N000 undo_finished sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5512025-06-04T01:30:05.531ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 0 saga_id = 50268305-c950-409c-9416-035f25ab5792
5522025-06-04T01:30:05.534ZDEBGnexus (SEC): saga log event new_state = N004 undo_started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5532025-06-04T01:30:05.534ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 4 saga_id = 50268305-c950-409c-9416-035f25ab5792
5542025-06-04T01:30:05.537ZDEBGnexus (SEC): saga log event new_state = N004 undo_finished sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5552025-06-04T01:30:05.537ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 4 saga_id = 50268305-c950-409c-9416-035f25ab5792
5562025-06-04T01:30:05.540ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 50268305-c950-409c-9416-035f25ab5792 sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
5572025-06-04T01:30:05.540ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 50268305-c950-409c-9416-035f25ab5792
5582025-06-04T01:30:05.546ZWARNnexus (SEC): saga finished action_error_node_name = "no_result1" action_error_source = ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} } file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1044 result = failure saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac undo_result = success
5592025-06-04T01:30:05.546ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 50268305-c950-409c-9416-035f25ab5792 saga_name = instance-delete saga_result = SagaResult { saga_id: 50268305-c950-409c-9416-035f25ab5792, saga_log: SagaLog { saga_id: 50268305-c950-409c-9416-035f25ab5792, unwinding: true, events: [N004 started, N004 succeeded, N000 started, N000 succeeded, N001 started, N001 failed, N000 undo_started, N000 undo_finished, N004 undo_started, N004 undo_finished], node_status: {0: UndoFinished, 1: Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }), 4: UndoFinished} }, kind: Err(SagaResultErr { error_node_name: "no_result1", error_source: ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }, undo_failure: None }) }
5602025-06-04T01:30:05.546ZINFOnexus (dropshot_external): request completed error_message_external = cannot delete instance: instance is running or has not yet fully stopped error_message_internal = cannot delete instance: instance is running or has not yet fully stopped (with internal context: saga ACTION error at node "no_result1") file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:855 latency_us = 408376 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = e061589b-eb0f-4f08-858b-b5428dc4ab61 response_code = 400 uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
5612025-06-04T01:30:06.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = d4892fe4-f69d-43f7-89a4-16c06494d616 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
5622025-06-04T01:30:06.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = d4892fe4-f69d-43f7-89a4-16c06494d616 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
5632025-06-04T01:30:06.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3057 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = d4892fe4-f69d-43f7-89a4-16c06494d616 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
5642025-06-04T01:30:06.548ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5652025-06-04T01:30:06.549ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5662025-06-04T01:30:06.550ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5672025-06-04T01:30:06.550ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5682025-06-04T01:30:06.552ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5692025-06-04T01:30:06.553ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
5702025-06-04T01:30:06.553ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5712025-06-04T01:30:06.553ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5722025-06-04T01:30:06.555ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5732025-06-04T01:30:06.555ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5742025-06-04T01:30:06.556ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5752025-06-04T01:30:06.557ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5762025-06-04T01:30:06.558ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5772025-06-04T01:30:06.558ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5782025-06-04T01:30:06.559ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5792025-06-04T01:30:06.561ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
5802025-06-04T01:30:06.561ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5812025-06-04T01:30:06.561ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5822025-06-04T01:30:06.562ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5832025-06-04T01:30:06.563ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
5842025-06-04T01:30:06.563ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5852025-06-04T01:30:06.564ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5862025-06-04T01:30:06.568ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5872025-06-04T01:30:06.568ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5882025-06-04T01:30:06.569ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5892025-06-04T01:30:06.569ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5902025-06-04T01:30:06.571ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5912025-06-04T01:30:06.571ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5922025-06-04T01:30:06.572ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5932025-06-04T01:30:06.573ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5942025-06-04T01:30:06.574ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5952025-06-04T01:30:06.575ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
5962025-06-04T01:30:06.575ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5972025-06-04T01:30:06.575ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5982025-06-04T01:30:06.576ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5992025-06-04T01:30:06.577ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6002025-06-04T01:30:06.578ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6012025-06-04T01:30:06.578ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6022025-06-04T01:30:06.579ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6032025-06-04T01:30:06.581ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
6042025-06-04T01:30:06.581ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6052025-06-04T01:30:06.581ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6062025-06-04T01:30:06.582ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6072025-06-04T01:30:06.582ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6082025-06-04T01:30:06.584ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6092025-06-04T01:30:06.584ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6102025-06-04T01:30:06.585ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6112025-06-04T01:30:06.585ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6122025-06-04T01:30:06.588ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6132025-06-04T01:30:06.588ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6142025-06-04T01:30:06.590ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6152025-06-04T01:30:06.590ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6162025-06-04T01:30:06.592ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6172025-06-04T01:30:06.593ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6182025-06-04T01:30:06.595ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6192025-06-04T01:30:06.598ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6202025-06-04T01:30:06.598ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6212025-06-04T01:30:06.598ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6222025-06-04T01:30:06.600ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6232025-06-04T01:30:06.600ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6242025-06-04T01:30:06.603ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6252025-06-04T01:30:06.603ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6262025-06-04T01:30:06.604ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6272025-06-04T01:30:06.605ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Database result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6282025-06-04T01:30:06.607ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6292025-06-04T01:30:06.610ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6302025-06-04T01:30:06.613ZINFOnexus (dropshot_external): set intended instance state actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true changed = false file = nexus/db-queries/src/db/datastore/instance.rs:812 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d intended_state = destroyed local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
6312025-06-04T01:30:06.613ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
6322025-06-04T01:30:06.613ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
6332025-06-04T01:30:06.614ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
6342025-06-04T01:30:06.615ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
6352025-06-04T01:30:06.615ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
6362025-06-04T01:30:06.616ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
6372025-06-04T01:30:06.619ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete
6382025-06-04T01:30:06.619ZINFOnexus (SEC): saga create dag = {"end_node":5,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,0,null],[3,5,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_delete.instance_delete_nat","label":"InstanceDeleteNat","name":"no_result4"}},{"Action":{"action_name":"instance_delete.instance_delete_record","label":"InstanceDeleteRecord","name":"no_result1"}},{"Action":{"action_name":"instance_delete.delete_network_interfaces","label":"DeleteNetworkInterfaces","name":"no_result2"}},{"Action":{"action_name":"instance_delete.deallocate_external_ip","label":"DeallocateExternalIp","name":"no_result3"}},{"Start":{"params":{"authz_instance":{"key":"5abad387-28b5-4eac-817c-81deb58a742d","lookup_type":{"ByName":"instance-506f220f14c"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ByName":"proj-25de54cd75b"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}},"boundary_switches":["switch0"],"instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":"c63ca8c6-5bef-4746-bdc3-504c0087eb2b","hostname":"localshark","identity":{"description":"","id":"5abad387-28b5-4eac-817c-81deb58a742d","name":"instance-506f220f14c","time_created":"2025-06-04T01:29:43.103614Z","time_deleted":null,"time_modified":"2025-06-04T01:30:03.817850Z"},"intended_state":"Destroyed","memory":1073741824,"ncpus":2,"project_id":"6a575fb0-c195-4d2a-ac26-16b83f49e832","runtime_state":{"dst_propolis_id":null,"gen":3,"migration_id":null,"nexus_state":"Vmm","propolis_id":"a8435d87-6398-4e8f-9fda-269bcd96ac6e","time_last_auto_restarted":null,"time_updated":"2025-06-04T01:29:43.103614Z"},"updater_gen":3,"updater_id":"a7bdc6e6-6858-4b38-be73-f70160601be1","user_data":[]},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","silo_user_id":"aa424c47-6907-47fb-925a-807ae77df887"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"instance-delete","start_node":4} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6392025-06-04T01:30:06.619ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete
6402025-06-04T01:30:06.624ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete
6412025-06-04T01:30:06.624ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6422025-06-04T01:30:06.624ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6432025-06-04T01:30:06.624ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
6442025-06-04T01:30:06.629ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6452025-06-04T01:30:06.629ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
6462025-06-04T01:30:06.633ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6472025-06-04T01:30:06.633ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
6482025-06-04T01:30:06.637ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6492025-06-04T01:30:06.637ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6502025-06-04T01:30:06.638ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6512025-06-04T01:30:06.639ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6522025-06-04T01:30:06.640ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6532025-06-04T01:30:06.640ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6542025-06-04T01:30:06.641ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6552025-06-04T01:30:06.642ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6562025-06-04T01:30:06.644ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6572025-06-04T01:30:06.644ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6582025-06-04T01:30:06.647ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6592025-06-04T01:30:06.647ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6602025-06-04T01:30:06.649ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6612025-06-04T01:30:06.649ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6622025-06-04T01:30:06.651ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6632025-06-04T01:30:06.654ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6642025-06-04T01:30:06.654ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d
6652025-06-04T01:30:06.654ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6662025-06-04T01:30:06.654ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6672025-06-04T01:30:06.656ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6682025-06-04T01:30:06.656ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6692025-06-04T01:30:06.658ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
6702025-06-04T01:30:06.658ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6712025-06-04T01:30:06.658ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteNat
6722025-06-04T01:30:06.660ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
6732025-06-04T01:30:06.660ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
6742025-06-04T01:30:06.660ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
6752025-06-04T01:30:06.661ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
6762025-06-04T01:30:06.662ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
6772025-06-04T01:30:06.663ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
6782025-06-04T01:30:06.663ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
6792025-06-04T01:30:06.666ZDEBGnexus: notifying dendrite of updates instance_id = Some(5abad387-28b5-4eac-817c-81deb58a742d (instance)) switch = switch0
6802025-06-04T01:30:06.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 760188812 } })
6812025-06-04T01:30:06.667ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1179
6822025-06-04T01:30:06.697ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
6832025-06-04T01:30:06.697ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6842025-06-04T01:30:06.697ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5439971-37ab-42c4-a7a4-7907e40a1755", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
6852025-06-04T01:30:06.697ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6862025-06-04T01:30:06.698ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6872025-06-04T01:30:06.730ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6882025-06-04T01:30:06.730ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
6892025-06-04T01:30:06.733ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
6902025-06-04T01:30:06.733ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
6912025-06-04T01:30:06.736ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6922025-06-04T01:30:06.736ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6932025-06-04T01:30:06.739ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6942025-06-04T01:30:06.739ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6952025-06-04T01:30:06.741ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6962025-06-04T01:30:06.741ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6972025-06-04T01:30:06.743ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6982025-06-04T01:30:06.743ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
6992025-06-04T01:30:06.746ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7002025-06-04T01:30:06.748ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7012025-06-04T01:30:06.748ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7022025-06-04T01:30:06.749ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7032025-06-04T01:30:06.827ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7042025-06-04T01:30:06.828ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7052025-06-04T01:30:06.833ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7062025-06-04T01:30:06.833ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7072025-06-04T01:30:06.851ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7082025-06-04T01:30:06.852ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7092025-06-04T01:30:06.864ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7102025-06-04T01:30:06.864ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = InstanceDeleteRecord
7112025-06-04T01:30:06.866ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7122025-06-04T01:30:06.866ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 1 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7132025-06-04T01:30:06.870ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7142025-06-04T01:30:06.870ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7152025-06-04T01:30:06.874ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7162025-06-04T01:30:06.874ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7172025-06-04T01:30:06.877ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7182025-06-04T01:30:06.877ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7192025-06-04T01:30:06.879ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7202025-06-04T01:30:06.879ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7212025-06-04T01:30:06.881ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7222025-06-04T01:30:06.882ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7232025-06-04T01:30:06.884ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7242025-06-04T01:30:06.887ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ByName("instance-506f220f14c") } result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7252025-06-04T01:30:06.887ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7262025-06-04T01:30:06.887ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeleteNetworkInterfaces
7272025-06-04T01:30:06.926ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7282025-06-04T01:30:06.926ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7292025-06-04T01:30:06.926ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Signaled
7302025-06-04T01:30:06.926ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7312025-06-04T01:30:06.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7322025-06-04T01:30:06.929ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7332025-06-04T01:30:06.929ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7342025-06-04T01:30:06.934ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeallocateExternalIp
7352025-06-04T01:30:06.934ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeallocateExternalIp
7362025-06-04T01:30:06.969ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7372025-06-04T01:30:06.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7382025-06-04T01:30:06.978ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7392025-06-04T01:30:06.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7402025-06-04T01:30:06.980ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7412025-06-04T01:30:06.982ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7422025-06-04T01:30:06.982ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7432025-06-04T01:30:06.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7442025-06-04T01:30:06.984ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7452025-06-04T01:30:06.985ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7462025-06-04T01:30:06.985ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7472025-06-04T01:30:06.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7482025-06-04T01:30:07.005ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeallocateExternalIp
7492025-06-04T01:30:07.005ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_node = DeallocateExternalIp
7502025-06-04T01:30:07.018ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7512025-06-04T01:30:07.018ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b295b031-dff6-4533-8174-9619a401fe1e", "content-length": "691", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
7522025-06-04T01:30:07.018ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
7532025-06-04T01:30:07.018ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
7542025-06-04T01:30:07.018ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
7552025-06-04T01:30:07.019ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 92.925228ms iteration = 26 status = {}
7562025-06-04T01:30:07.072ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7572025-06-04T01:30:07.072ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7582025-06-04T01:30:07.077ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7592025-06-04T01:30:07.077ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7602025-06-04T01:30:07.080ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7612025-06-04T01:30:07.080ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7622025-06-04T01:30:07.084ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7632025-06-04T01:30:07.084ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a
7642025-06-04T01:30:07.122ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
7652025-06-04T01:30:07.122ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a saga_name = instance-delete saga_result = SagaResult { saga_id: 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a, saga_log: SagaLog { saga_id: 289aeb5f-f909-4b6d-9abc-6cf1a9f3d84a, unwinding: false, events: [N004 started, N004 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N005 started, N005 succeeded], node_status: {0: Succeeded(Null), 1: Succeeded(Null), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"no_result1": Null, "no_result2": Null, "no_result3": Null, "no_result4": Null} }) }
7662025-06-04T01:30:07.122ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Signaled
7672025-06-04T01:30:07.122ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
7682025-06-04T01:30:07.122ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7692025-06-04T01:30:07.122ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 573935 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 5a0b4872-5667-471a-b6ce-85754124154b response_code = 204 uri = /v1/instances/instance-506f220f14c?project=proj-25de54cd75b
7702025-06-04T01:30:07.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7712025-06-04T01:30:07.123ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7722025-06-04T01:30:07.123ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7732025-06-04T01:30:07.124ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7742025-06-04T01:30:07.125ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7752025-06-04T01:30:07.126ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7762025-06-04T01:30:07.126ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7772025-06-04T01:30:07.126ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7782025-06-04T01:30:07.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7792025-06-04T01:30:07.127ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7802025-06-04T01:30:07.127ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7812025-06-04T01:30:07.128ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
7822025-06-04T01:30:07.128ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7832025-06-04T01:30:07.129ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7842025-06-04T01:30:07.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7852025-06-04T01:30:07.129ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7862025-06-04T01:30:07.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7872025-06-04T01:30:07.130ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7882025-06-04T01:30:07.130ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7892025-06-04T01:30:07.131ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7902025-06-04T01:30:07.132ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7912025-06-04T01:30:07.133ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7922025-06-04T01:30:07.133ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7932025-06-04T01:30:07.135ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7942025-06-04T01:30:07.136ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
7952025-06-04T01:30:07.136ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7962025-06-04T01:30:07.136ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7972025-06-04T01:30:07.137ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7982025-06-04T01:30:07.139ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
7992025-06-04T01:30:07.139ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8002025-06-04T01:30:07.139ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8012025-06-04T01:30:07.143ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8022025-06-04T01:30:07.143ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8032025-06-04T01:30:07.144ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8042025-06-04T01:30:07.144ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8052025-06-04T01:30:07.146ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8062025-06-04T01:30:07.146ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8072025-06-04T01:30:07.147ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8082025-06-04T01:30:07.147ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8092025-06-04T01:30:07.149ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8102025-06-04T01:30:07.150ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
8112025-06-04T01:30:07.150ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8122025-06-04T01:30:07.150ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8132025-06-04T01:30:07.157ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8142025-06-04T01:30:07.157ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8152025-06-04T01:30:07.159ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8162025-06-04T01:30:07.159ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8172025-06-04T01:30:07.160ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8182025-06-04T01:30:07.162ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
8192025-06-04T01:30:07.162ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8202025-06-04T01:30:07.162ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8212025-06-04T01:30:07.163ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8222025-06-04T01:30:07.163ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8232025-06-04T01:30:07.165ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8242025-06-04T01:30:07.165ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8252025-06-04T01:30:07.167ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8262025-06-04T01:30:07.167ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8272025-06-04T01:30:07.169ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8282025-06-04T01:30:07.170ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8292025-06-04T01:30:07.172ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8302025-06-04T01:30:07.172ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8312025-06-04T01:30:07.174ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8322025-06-04T01:30:07.174ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8332025-06-04T01:30:07.176ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8342025-06-04T01:30:07.179ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: c63ca8c6-5bef-4746-bdc3-504c0087eb2b, lookup_type: ByName("disk-b287276514b") } result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8352025-06-04T01:30:07.179ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8362025-06-04T01:30:07.179ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8372025-06-04T01:30:07.181ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8382025-06-04T01:30:07.181ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8392025-06-04T01:30:07.182ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8402025-06-04T01:30:07.183ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8412025-06-04T01:30:07.184ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8422025-06-04T01:30:07.184ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8432025-06-04T01:30:07.186ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8442025-06-04T01:30:07.186ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8452025-06-04T01:30:07.189ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8462025-06-04T01:30:07.189ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8472025-06-04T01:30:07.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = b8d0fdf6-4a99-4a9b-9bd3-d10e3a59a777 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
8482025-06-04T01:30:07.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = b8d0fdf6-4a99-4a9b-9bd3-d10e3a59a777 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
8492025-06-04T01:30:07.196ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2916 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:60994 req_id = b8d0fdf6-4a99-4a9b-9bd3-d10e3a59a777 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
8502025-06-04T01:30:07.224ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8512025-06-04T01:30:07.224ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Database result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8522025-06-04T01:30:07.227ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8532025-06-04T01:30:07.230ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: c63ca8c6-5bef-4746-bdc3-504c0087eb2b, lookup_type: ById(c63ca8c6-5bef-4746-bdc3-504c0087eb2b) } result = Ok(()) uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
8542025-06-04T01:30:07.230ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
8552025-06-04T01:30:07.230ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,5,null],[4,6,null],[6,7,null],[5,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[3,4,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"disk_delete.delete_disk_record","label":"DeleteDiskRecord","name":"deleted_disk"}},{"Action":{"action_name":"disk_delete.space_account","label":"SpaceAccount","name":"no_result1"}},{"Constant":{"name":"params_for_volume_delete_subsaga","value":{"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","silo_user_id":"aa424c47-6907-47fb-925a-807ae77df887"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"8f2364fd-83e5-47f9-873c-31c48ff55f3b"}}},{"SubsagaStart":{"params_node_name":"params_for_volume_delete_subsaga","saga_name":"volume-delete"}},{"Action":{"action_name":"volume_delete.decrease_crucible_resource_count","label":"DecreaseCrucibleResourceCount","name":"crucible_resources_to_delete"}},{"Action":{"action_name":"volume_delete.delete_crucible_regions","label":"DeleteCrucibleRegions","name":"no_result_1"}},{"Action":{"action_name":"volume_delete.delete_crucible_running_snapshots","label":"DeleteCrucibleRunningSnapshots","name":"no_result_2"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshots","label":"DeleteCrucibleSnapshots","name":"no_result_3"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshot_records","label":"DeleteCrucibleSnapshotRecords","name":"no_result_4"}},{"Action":{"action_name":"volume_delete.find_freed_crucible_regions","label":"FindFreedCrucibleRegions","name":"freed_crucible_regions"}},{"Action":{"action_name":"volume_delete.delete_freed_crucible_regions","label":"DeleteFreedCrucibleRegions","name":"no_result_5"}},{"Action":{"action_name":"volume_delete.hard_delete_volume_record","label":"HardDeleteVolumeRecord","name":"volume_hard_deleted"}},{"SubsagaEnd":{"name":"volume_delete_subsaga_no_result"}},{"Start":{"params":{"disk_id":"c63ca8c6-5bef-4746-bdc3-504c0087eb2b","project_id":"6a575fb0-c195-4d2a-ac26-16b83f49e832","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","silo_user_id":"aa424c47-6907-47fb-925a-807ae77df887"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"8f2364fd-83e5-47f9-873c-31c48ff55f3b"}}},"End"]},"saga_name":"disk-delete","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8562025-06-04T01:30:07.230ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
8572025-06-04T01:30:07.232ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8582025-06-04T01:30:07.232ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8592025-06-04T01:30:07.232ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e8f5915-85d4-4f95-a352-318e6d51502e", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
8602025-06-04T01:30:07.232ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8612025-06-04T01:30:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8622025-06-04T01:30:07.242ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
8632025-06-04T01:30:07.242ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8642025-06-04T01:30:07.242ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8652025-06-04T01:30:07.242ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8662025-06-04T01:30:07.252ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8672025-06-04T01:30:07.252ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8682025-06-04T01:30:07.258ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8692025-06-04T01:30:07.258ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8702025-06-04T01:30:07.278ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8712025-06-04T01:30:07.278ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"block_size": String("Traditional"), "create_image_id": String("d332093d-fb21-4c21-b11a-d78093fb9f43"), "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("c63ca8c6-5bef-4746-bdc3-504c0087eb2b"), "name": String("disk-b287276514b"), "time_created": String("2025-06-04T01:29:41.700952Z"), "time_deleted": Null, "time_modified": String("2025-06-04T01:29:41.700952Z")}, "pantry_address": Null, "project_id": String("6a575fb0-c195-4d2a-ac26-16b83f49e832"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2025-06-04T01:29:42.896328Z")}, "size": Number(2147483648), "slot": Null, "volume_id": String("8f2364fd-83e5-47f9-873c-31c48ff55f3b")}) node_id = 0 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8722025-06-04T01:30:07.279ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8732025-06-04T01:30:07.279ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8742025-06-04T01:30:07.279ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c9ccb074-d8a5-4cff-a26e-c416f1194db2", "date": "Wed, 04 Jun 2025 01:30:07 GMT"} })
8752025-06-04T01:30:07.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8762025-06-04T01:30:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8772025-06-04T01:30:07.282ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8782025-06-04T01:30:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8792025-06-04T01:30:07.284ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8802025-06-04T01:30:07.284ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8812025-06-04T01:30:07.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8822025-06-04T01:30:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8832025-06-04T01:30:07.288ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
8842025-06-04T01:30:07.288ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8852025-06-04T01:30:07.288ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "693fbee7-2425-4e11-94e7-18e666ebb855", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
8862025-06-04T01:30:07.289ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete saga_node = SpaceAccount
8872025-06-04T01:30:07.289ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete saga_node = SpaceAccount
8882025-06-04T01:30:07.290ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 167.881932ms iteration = 27 status = {}
8892025-06-04T01:30:07.307ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8902025-06-04T01:30:07.307ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 1 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8912025-06-04T01:30:07.310ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8922025-06-04T01:30:07.310ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8932025-06-04T01:30:07.313ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8942025-06-04T01:30:07.313ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"), "silo_user_id": String("aa424c47-6907-47fb-925a-807ae77df887")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("8f2364fd-83e5-47f9-873c-31c48ff55f3b")}) node_id = 2 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8952025-06-04T01:30:07.326ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8962025-06-04T01:30:07.326ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8972025-06-04T01:30:07.333ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
8982025-06-04T01:30:07.333ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
8992025-06-04T01:30:07.340ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9002025-06-04T01:30:07.340ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9012025-06-04T01:30:07.417ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9022025-06-04T01:30:07.417ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("4ff4218c-33d5-4201-8a68-9a0a936c42d1"), String("22f4c290-f5a6-4b5d-abf7-5b0f61bb267a"), String("594eea1c-5bf0-400b-884c-d4a9425449ce")]}}) node_id = 4 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9032025-06-04T01:30:07.421ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9042025-06-04T01:30:07.421ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9052025-06-04T01:30:07.424ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9062025-06-04T01:30:07.424ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9072025-06-04T01:30:07.429ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9082025-06-04T01:30:07.429ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9092025-06-04T01:30:07.462ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 72.153279ms file = nexus/src/app/crucible.rs:864 region_id = 4ff4218c-33d5-4201-8a68-9a0a936c42d1 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9102025-06-04T01:30:07.466ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 63.961245ms file = nexus/src/app/crucible.rs:864 region_id = 594eea1c-5bf0-400b-884c-d4a9425449ce saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9112025-06-04T01:30:07.466ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 120.785097ms file = nexus/src/app/crucible.rs:864 region_id = 22f4c290-f5a6-4b5d-abf7-5b0f61bb267a saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9122025-06-04T01:30:07.535ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 140.255216ms file = nexus/src/app/crucible.rs:864 region_id = 594eea1c-5bf0-400b-884c-d4a9425449ce saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9132025-06-04T01:30:07.540ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 147.918242ms file = nexus/src/app/crucible.rs:864 region_id = 4ff4218c-33d5-4201-8a68-9a0a936c42d1 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9142025-06-04T01:30:07.592ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:807 region_id = 22f4c290-f5a6-4b5d-abf7-5b0f61bb267a saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9152025-06-04T01:30:07.680ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:807 region_id = 594eea1c-5bf0-400b-884c-d4a9425449ce saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9162025-06-04T01:30:07.692ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:807 region_id = 4ff4218c-33d5-4201-8a68-9a0a936c42d1 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9172025-06-04T01:30:07.730ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9182025-06-04T01:30:07.730ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9192025-06-04T01:30:07.734ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9202025-06-04T01:30:07.734ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9212025-06-04T01:30:07.739ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9222025-06-04T01:30:07.739ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 7 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9232025-06-04T01:30:07.743ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9242025-06-04T01:30:07.743ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9252025-06-04T01:30:07.747ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9262025-06-04T01:30:07.747ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9272025-06-04T01:30:07.751ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9282025-06-04T01:30:07.751ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9292025-06-04T01:30:07.763ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9302025-06-04T01:30:07.763ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"datasets_and_regions": Array [], "volumes": Array []}) node_id = 9 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9312025-06-04T01:30:07.767ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9322025-06-04T01:30:07.767ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9332025-06-04T01:30:07.772ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9342025-06-04T01:30:07.772ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9352025-06-04T01:30:07.776ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9362025-06-04T01:30:07.776ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9372025-06-04T01:30:07.784ZINFOnexus (SagaExecutor): allocated regions for 8f2364fd-83e5-47f9-873c-31c48ff55f3b is empty, calling volume_hard_delete file = nexus/src/app/sagas/volume_delete.rs:523 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete
9382025-06-04T01:30:07.789ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9392025-06-04T01:30:07.789ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Bool(true)) node_id = 11 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9402025-06-04T01:30:07.794ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9412025-06-04T01:30:07.794ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9422025-06-04T01:30:07.799ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9432025-06-04T01:30:07.799ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Bool(true)) node_id = 12 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9442025-06-04T01:30:07.802ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9452025-06-04T01:30:07.802ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9462025-06-04T01:30:07.806ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9472025-06-04T01:30:07.806ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9482025-06-04T01:30:07.810ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9492025-06-04T01:30:07.810ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c
9502025-06-04T01:30:07.819ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
9512025-06-04T01:30:07.820ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 924c8f04-1878-419b-9852-33ebbaf04c6c saga_name = disk-delete saga_result = SagaResult { saga_id: 924c8f04-1878-419b-9852-33ebbaf04c6c, saga_log: SagaLog { saga_id: 924c8f04-1878-419b-9852-33ebbaf04c6c, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N006 started, N006 succeeded, N005 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(Object {"block_size": String("Traditional"), "create_image_id": String("d332093d-fb21-4c21-b11a-d78093fb9f43"), "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("c63ca8c6-5bef-4746-bdc3-504c0087eb2b"), "name": String("disk-b287276514b"), "time_created": String("2025-06-04T01:29:41.700952Z"), "time_deleted": Null, "time_modified": String("2025-06-04T01:29:41.700952Z")}, "pantry_address": Null, "project_id": String("6a575fb0-c195-4d2a-ac26-16b83f49e832"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2025-06-04T01:29:42.896328Z")}, "size": Number(2147483648), "slot": Null, "volume_id": String("8f2364fd-83e5-47f9-873c-31c48ff55f3b")}), 1: Succeeded(Null), 2: Succeeded(Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"), "silo_user_id": String("aa424c47-6907-47fb-925a-807ae77df887")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("8f2364fd-83e5-47f9-873c-31c48ff55f3b")}), 3: Succeeded(Null), 4: Succeeded(Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("4ff4218c-33d5-4201-8a68-9a0a936c42d1"), String("22f4c290-f5a6-4b5d-abf7-5b0f61bb267a"), String("594eea1c-5bf0-400b-884c-d4a9425449ce")]}}), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Null), 8: Succeeded(Null), 9: Succeeded(Object {"datasets_and_regions": Array [], "volumes": Array []}), 10: Succeeded(Null), 11: Succeeded(Bool(true)), 12: Succeeded(Bool(true)), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Bool(true), node_outputs: {"crucible_resources_to_delete": Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("4ff4218c-33d5-4201-8a68-9a0a936c42d1"), String("22f4c290-f5a6-4b5d-abf7-5b0f61bb267a"), String("594eea1c-5bf0-400b-884c-d4a9425449ce")]}}, "deleted_disk": Object {"block_size": String("Traditional"), "create_image_id": String("d332093d-fb21-4c21-b11a-d78093fb9f43"), "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("c63ca8c6-5bef-4746-bdc3-504c0087eb2b"), "name": String("disk-b287276514b"), "time_created": String("2025-06-04T01:29:41.700952Z"), "time_deleted": Null, "time_modified": String("2025-06-04T01:29:41.700952Z")}, "pantry_address": Null, "project_id": String("6a575fb0-c195-4d2a-ac26-16b83f49e832"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2025-06-04T01:29:42.896328Z")}, "size": Number(2147483648), "slot": Null, "volume_id": String("8f2364fd-83e5-47f9-873c-31c48ff55f3b")}, "freed_crucible_regions": Object {"datasets_and_regions": Array [], "volumes": Array []}, "no_result1": Null, "no_result_1": Null, "no_result_2": Null, "no_result_3": Null, "no_result_4": Null, "no_result_5": Null, "params_for_volume_delete_subsaga": Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"), "silo_user_id": String("aa424c47-6907-47fb-925a-807ae77df887")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("8f2364fd-83e5-47f9-873c-31c48ff55f3b")}, "volume_delete_subsaga_no_result": Bool(true), "volume_hard_deleted": Bool(true)} }) }
9522025-06-04T01:30:07.820ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 696542 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = a14ded78-977b-4674-9026-31378df32ee0 response_code = 204 uri = /v1/disks/disk-b287276514b?project=proj-25de54cd75b
9532025-06-04T01:30:07.821ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9542025-06-04T01:30:07.822ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9552025-06-04T01:30:07.824ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9562025-06-04T01:30:07.824ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9572025-06-04T01:30:07.826ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9582025-06-04T01:30:07.828ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
9592025-06-04T01:30:07.828ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9602025-06-04T01:30:07.829ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9612025-06-04T01:30:07.831ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9622025-06-04T01:30:07.832ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9632025-06-04T01:30:07.834ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9642025-06-04T01:30:07.834ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9652025-06-04T01:30:07.836ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9662025-06-04T01:30:07.836ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9672025-06-04T01:30:07.839ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9682025-06-04T01:30:07.841ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
9692025-06-04T01:30:07.841ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9702025-06-04T01:30:07.841ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9712025-06-04T01:30:07.844ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9722025-06-04T01:30:07.846ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
9732025-06-04T01:30:07.846ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9742025-06-04T01:30:07.846ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9752025-06-04T01:30:07.851ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9762025-06-04T01:30:07.852ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9772025-06-04T01:30:07.853ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9782025-06-04T01:30:07.853ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9792025-06-04T01:30:07.854ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9802025-06-04T01:30:07.855ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9812025-06-04T01:30:07.856ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9822025-06-04T01:30:07.856ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9832025-06-04T01:30:07.857ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9842025-06-04T01:30:07.859ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
9852025-06-04T01:30:07.859ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9862025-06-04T01:30:07.859ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9872025-06-04T01:30:07.860ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9882025-06-04T01:30:07.860ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9892025-06-04T01:30:07.861ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9902025-06-04T01:30:07.861ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9912025-06-04T01:30:07.862ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9922025-06-04T01:30:07.863ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
9932025-06-04T01:30:07.863ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
9942025-06-04T01:30:07.864ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
9952025-06-04T01:30:07.865ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
9962025-06-04T01:30:07.865ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
9972025-06-04T01:30:07.867ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
9982025-06-04T01:30:07.867ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
9992025-06-04T01:30:07.869ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10002025-06-04T01:30:07.869ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10012025-06-04T01:30:07.871ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10022025-06-04T01:30:07.871ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10032025-06-04T01:30:07.873ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10042025-06-04T01:30:07.873ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10052025-06-04T01:30:07.875ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10062025-06-04T01:30:07.875ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10072025-06-04T01:30:07.877ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10082025-06-04T01:30:07.877ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10092025-06-04T01:30:07.880ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10102025-06-04T01:30:07.883ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: a169ce39-99d1-4d24-a380-5ee40770240e, lookup_type: ByName("default") }, key: 300faf8a-aa77-4b7e-9f3e-03c9259ec68f, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10112025-06-04T01:30:07.883ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete
10122025-06-04T01:30:07.883ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"vpc_subnet_delete.vpc_subnet_delete_subnet","label":"VpcSubnetDeleteSubnet","name":"no_result_0"}},{"Action":{"action_name":"vpc_subnet_delete.vpc_subnet_delete_sys_route","label":"VpcSubnetDeleteSysRoute","name":"no_result_1"}},{"Action":{"action_name":"vpc_subnet_delete.vpc_notify_rpw","label":"VpcNotifyRpw","name":"no_result_2"}},{"Start":{"params":{"authz_subnet":{"key":"300faf8a-aa77-4b7e-9f3e-03c9259ec68f","lookup_type":{"ByName":"default"},"parent":{"key":"a169ce39-99d1-4d24-a380-5ee40770240e","lookup_type":{"ByName":"default"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ByName":"proj-25de54cd75b"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}}},"authz_vpc":{"key":"a169ce39-99d1-4d24-a380-5ee40770240e","lookup_type":{"ByName":"default"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ByName":"proj-25de54cd75b"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}},"db_subnet":{"custom_router_id":null,"identity":{"description":"The default subnet for default","id":"300faf8a-aa77-4b7e-9f3e-03c9259ec68f","name":"default","time_created":"2025-06-04T01:29:40.925569Z","time_deleted":null,"time_modified":"2025-06-04T01:29:40.925569Z"},"ipv4_block":"172.30.0.0/22","ipv6_block":"fd9a:9de:f653::/64","rcgen":2,"vpc_id":"a169ce39-99d1-4d24-a380-5ee40770240e"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","silo_user_id":"aa424c47-6907-47fb-925a-807ae77df887"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"vpc-subnet-delete","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10132025-06-04T01:30:07.883ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete
10142025-06-04T01:30:07.887ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete
10152025-06-04T01:30:07.887ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10162025-06-04T01:30:07.887ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10172025-06-04T01:30:07.887ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10182025-06-04T01:30:07.891ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10192025-06-04T01:30:07.891ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10202025-06-04T01:30:07.894ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10212025-06-04T01:30:07.894ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10222025-06-04T01:30:07.897ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10232025-06-04T01:30:07.897ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10242025-06-04T01:30:07.899ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10252025-06-04T01:30:07.899ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10262025-06-04T01:30:07.902ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10272025-06-04T01:30:07.902ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10282025-06-04T01:30:07.904ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10292025-06-04T01:30:07.904ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10302025-06-04T01:30:07.906ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10312025-06-04T01:30:07.909ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: a169ce39-99d1-4d24-a380-5ee40770240e, lookup_type: ByName("default") }, key: 300faf8a-aa77-4b7e-9f3e-03c9259ec68f, lookup_type: ByName("default") } result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10322025-06-04T01:30:07.909ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10332025-06-04T01:30:07.909ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSubnet
10342025-06-04T01:30:07.916ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10352025-06-04T01:30:07.916ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10362025-06-04T01:30:07.919ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10372025-06-04T01:30:07.919ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10382025-06-04T01:30:07.922ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSysRoute
10392025-06-04T01:30:07.922ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcSubnetDeleteSysRoute
10402025-06-04T01:30:07.957ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10412025-06-04T01:30:07.957ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 1 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10422025-06-04T01:30:07.961ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10432025-06-04T01:30:07.961ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10442025-06-04T01:30:07.964ZDEBGnexus (SagaExecutor): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true roles = RoleSet { roles: {} } saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcNotifyRpw
10452025-06-04T01:30:07.964ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true resource = Database result = Ok(()) saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_node = VpcNotifyRpw
10462025-06-04T01:30:07.969ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10472025-06-04T01:30:07.969ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10482025-06-04T01:30:07.973ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10492025-06-04T01:30:07.973ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10502025-06-04T01:30:07.976ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10512025-06-04T01:30:07.976ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10522025-06-04T01:30:07.979ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10532025-06-04T01:30:07.979ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52
10542025-06-04T01:30:08.017ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete sec_id = 34942533-fcec-49f1-bc91-3bae11367eac
10552025-06-04T01:30:08.017ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 1a649639-30a5-40df-94e5-19f5f4560f52 saga_name = vpc-subnet-delete saga_result = SagaResult { saga_id: 1a649639-30a5-40df-94e5-19f5f4560f52, saga_log: SagaLog { saga_id: 1a649639-30a5-40df-94e5-19f5f4560f52, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(Null), 1: Succeeded(Null), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"no_result_0": Null, "no_result_1": Null, "no_result_2": Null} }) }
10562025-06-04T01:30:08.017ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 195793 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 48dfd182-899b-4c75-a6bf-1a1460a30193 response_code = 204 uri = /v1/vpc-subnets/default?project=proj-25de54cd75b&vpc=default
10572025-06-04T01:30:08.017ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Signaled
10582025-06-04T01:30:08.017ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10592025-06-04T01:30:08.017ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10602025-06-04T01:30:08.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10612025-06-04T01:30:08.018ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10622025-06-04T01:30:08.018ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10632025-06-04T01:30:08.019ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10642025-06-04T01:30:08.020ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10652025-06-04T01:30:08.021ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10662025-06-04T01:30:08.021ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10672025-06-04T01:30:08.021ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10682025-06-04T01:30:08.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10692025-06-04T01:30:08.022ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
10702025-06-04T01:30:08.023ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10712025-06-04T01:30:08.023ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
10722025-06-04T01:30:08.023ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10732025-06-04T01:30:08.023ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10742025-06-04T01:30:08.024ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10752025-06-04T01:30:08.025ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10762025-06-04T01:30:08.025ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10772025-06-04T01:30:08.025ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10782025-06-04T01:30:08.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10792025-06-04T01:30:08.026ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10802025-06-04T01:30:08.026ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10812025-06-04T01:30:08.027ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10822025-06-04T01:30:08.027ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10832025-06-04T01:30:08.029ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
10842025-06-04T01:30:08.030ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
10852025-06-04T01:30:08.030ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10862025-06-04T01:30:08.030ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10872025-06-04T01:30:08.031ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
10882025-06-04T01:30:08.033ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
10892025-06-04T01:30:08.033ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10902025-06-04T01:30:08.033ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10912025-06-04T01:30:08.037ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10922025-06-04T01:30:08.038ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10932025-06-04T01:30:08.039ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10942025-06-04T01:30:08.039ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10952025-06-04T01:30:08.041ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10962025-06-04T01:30:08.041ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10972025-06-04T01:30:08.042ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10982025-06-04T01:30:08.043ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10992025-06-04T01:30:08.044ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
11002025-06-04T01:30:08.045ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
11012025-06-04T01:30:08.045ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11022025-06-04T01:30:08.045ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11032025-06-04T01:30:08.046ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11042025-06-04T01:30:08.046ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11052025-06-04T01:30:08.048ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11062025-06-04T01:30:08.048ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11072025-06-04T01:30:08.049ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
11082025-06-04T01:30:08.050ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
11092025-06-04T01:30:08.050ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11102025-06-04T01:30:08.051ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11112025-06-04T01:30:08.052ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11122025-06-04T01:30:08.052ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11132025-06-04T01:30:08.054ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11142025-06-04T01:30:08.054ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11152025-06-04T01:30:08.056ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11162025-06-04T01:30:08.056ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11172025-06-04T01:30:08.059ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11182025-06-04T01:30:08.059ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11192025-06-04T01:30:08.061ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11202025-06-04T01:30:08.061ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11212025-06-04T01:30:08.064ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11222025-06-04T01:30:08.064ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11232025-06-04T01:30:08.066ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11242025-06-04T01:30:08.069ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: a169ce39-99d1-4d24-a380-5ee40770240e, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11252025-06-04T01:30:08.069ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11262025-06-04T01:30:08.069ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11272025-06-04T01:30:08.072ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11282025-06-04T01:30:08.072ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11292025-06-04T01:30:08.074ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11302025-06-04T01:30:08.074ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11312025-06-04T01:30:08.076ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11322025-06-04T01:30:08.076ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11332025-06-04T01:30:08.079ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11342025-06-04T01:30:08.081ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: a169ce39-99d1-4d24-a380-5ee40770240e, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11352025-06-04T01:30:08.081ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11362025-06-04T01:30:08.081ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11372025-06-04T01:30:08.083ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11382025-06-04T01:30:08.083ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11392025-06-04T01:30:08.088ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11402025-06-04T01:30:08.088ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11412025-06-04T01:30:08.117ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
11422025-06-04T01:30:08.117ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11432025-06-04T01:30:08.117ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fad9cf40-008f-44f8-97f8-aebc1d3a83ab", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:08 GMT"} })
11442025-06-04T01:30:08.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11452025-06-04T01:30:08.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11462025-06-04T01:30:08.119ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11472025-06-04T01:30:08.119ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11482025-06-04T01:30:08.122ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11492025-06-04T01:30:08.122ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11502025-06-04T01:30:08.124ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11512025-06-04T01:30:08.124ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11522025-06-04T01:30:08.126ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11532025-06-04T01:30:08.130ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: a169ce39-99d1-4d24-a380-5ee40770240e, lookup_type: ByName("default") }, key: be760f9c-74b6-409c-937e-536de4f9e782, lookup_type: ById(be760f9c-74b6-409c-937e-536de4f9e782) } result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11542025-06-04T01:30:08.130ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11552025-06-04T01:30:08.130ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11562025-06-04T01:30:08.131ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
11572025-06-04T01:30:08.131ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11582025-06-04T01:30:08.132ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "83ac3c97-7af2-430a-99c4-779eece05039", "date": "Wed, 04 Jun 2025 01:30:08 GMT"} })
11592025-06-04T01:30:08.132ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11602025-06-04T01:30:08.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11612025-06-04T01:30:08.133ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11622025-06-04T01:30:08.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11632025-06-04T01:30:08.134ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11642025-06-04T01:30:08.134ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11652025-06-04T01:30:08.135ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11662025-06-04T01:30:08.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11672025-06-04T01:30:08.139ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
11682025-06-04T01:30:08.139ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11692025-06-04T01:30:08.139ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "958de46a-ed70-4955-b4fe-b38f157300de", "date": "Wed, 04 Jun 2025 01:30:08 GMT"} })
11702025-06-04T01:30:08.140ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 123.310132ms iteration = 28 status = {}
11712025-06-04T01:30:08.142ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11722025-06-04T01:30:08.142ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11732025-06-04T01:30:08.144ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11742025-06-04T01:30:08.145ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11752025-06-04T01:30:08.146ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11762025-06-04T01:30:08.146ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11772025-06-04T01:30:08.149ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11782025-06-04T01:30:08.149ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11792025-06-04T01:30:08.151ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11802025-06-04T01:30:08.151ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11812025-06-04T01:30:08.154ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11822025-06-04T01:30:08.156ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") }, key: a169ce39-99d1-4d24-a380-5ee40770240e, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11832025-06-04T01:30:08.156ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-25de54cd75b
11842025-06-04T01:30:08.156ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-25de54cd75b
11852025-06-04T01:30:08.162ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 143712 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = 7f0fea03-37e1-4cda-8bb0-502b4403c276 response_code = 204 uri = /v1/vpcs/default?project=proj-25de54cd75b
11862025-06-04T01:30:08.163ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11872025-06-04T01:30:08.163ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11882025-06-04T01:30:08.164ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11892025-06-04T01:30:08.164ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11902025-06-04T01:30:08.166ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
11912025-06-04T01:30:08.167ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
11922025-06-04T01:30:08.167ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11932025-06-04T01:30:08.167ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11942025-06-04T01:30:08.168ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11952025-06-04T01:30:08.168ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11962025-06-04T01:30:08.169ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11972025-06-04T01:30:08.169ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11982025-06-04T01:30:08.171ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11992025-06-04T01:30:08.171ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12002025-06-04T01:30:08.172ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12012025-06-04T01:30:08.173ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
12022025-06-04T01:30:08.173ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12032025-06-04T01:30:08.173ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12042025-06-04T01:30:08.174ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12052025-06-04T01:30:08.176ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3 (console_session), lookup_type: ById(a5c6d658-7d1c-40a2-a8bd-5ab728a65ea3) } result = Ok(())
12062025-06-04T01:30:08.176ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12072025-06-04T01:30:08.176ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12082025-06-04T01:30:08.180ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12092025-06-04T01:30:08.181ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12102025-06-04T01:30:08.182ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12112025-06-04T01:30:08.182ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12122025-06-04T01:30:08.183ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12132025-06-04T01:30:08.183ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12142025-06-04T01:30:08.184ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12152025-06-04T01:30:08.185ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12162025-06-04T01:30:08.186ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12172025-06-04T01:30:08.187ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: aa424c47-6907-47fb-925a-807ae77df887, lookup_type: ById(aa424c47-6907-47fb-925a-807ae77df887) } result = Ok(())
12182025-06-04T01:30:08.187ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12192025-06-04T01:30:08.187ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12202025-06-04T01:30:08.189ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12212025-06-04T01:30:08.189ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12222025-06-04T01:30:08.190ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12232025-06-04T01:30:08.190ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12242025-06-04T01:30:08.192ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12252025-06-04T01:30:08.193ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) } result = Ok(())
12262025-06-04T01:30:08.193ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12272025-06-04T01:30:08.193ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12282025-06-04T01:30:08.194ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12292025-06-04T01:30:08.195ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12302025-06-04T01:30:08.196ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12312025-06-04T01:30:08.196ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12322025-06-04T01:30:08.198ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12332025-06-04T01:30:08.199ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12342025-06-04T01:30:08.201ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12352025-06-04T01:30:08.201ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12362025-06-04T01:30:08.203ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12372025-06-04T01:30:08.204ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12382025-06-04T01:30:08.205ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/projects/proj-25de54cd75b
12392025-06-04T01:30:08.207ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") } result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12402025-06-04T01:30:08.207ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12412025-06-04T01:30:08.207ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12422025-06-04T01:30:08.210ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12432025-06-04T01:30:08.210ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12442025-06-04T01:30:08.212ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12452025-06-04T01:30:08.212ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12462025-06-04T01:30:08.214ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12472025-06-04T01:30:08.214ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12482025-06-04T01:30:08.217ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {(Silo, a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, "admin")} } uri = /v1/projects/proj-25de54cd75b
12492025-06-04T01:30:08.219ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ByName("proj-25de54cd75b") } result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12502025-06-04T01:30:08.219ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12512025-06-04T01:30:08.219ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12522025-06-04T01:30:08.221ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12532025-06-04T01:30:08.221ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12542025-06-04T01:30:08.223ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12552025-06-04T01:30:08.223ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12562025-06-04T01:30:08.234ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12572025-06-04T01:30:08.234ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12582025-06-04T01:30:08.240ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12592025-06-04T01:30:08.240ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12602025-06-04T01:30:08.242ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12612025-06-04T01:30:08.242ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12622025-06-04T01:30:08.244ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12632025-06-04T01:30:08.244ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12642025-06-04T01:30:08.246ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12652025-06-04T01:30:08.246ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12662025-06-04T01:30:08.248ZDEBGnexus (dropshot_external): roles actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 roles = RoleSet { roles: {} } uri = /v1/projects/proj-25de54cd75b
12672025-06-04T01:30:08.248ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: aa424c47-6907-47fb-925a-807ae77df887, silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, .. }) actor_id = aa424c47-6907-47fb-925a-807ae77df887 authenticated = true local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 resource = Database result = Ok(()) uri = /v1/projects/proj-25de54cd75b
12682025-06-04T01:30:08.258ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 95408 local_addr = 172.30.2.7:443 method = DELETE remote_addr = 10.151.2.100:45663 req_id = ffccda03-93dc-492f-be67-670ef3bdbe19 response_code = 204 uri = /v1/projects/proj-25de54cd75b
12692025-06-04T01:30:09.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
12702025-06-04T01:30:09.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
12712025-06-04T01:30:09.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12722025-06-04T01:30:09.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12732025-06-04T01:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12742025-06-04T01:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12752025-06-04T01:30:09.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12762025-06-04T01:30:09.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12772025-06-04T01:30:09.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12782025-06-04T01:30:09.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12792025-06-04T01:30:09.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12802025-06-04T01:30:09.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12812025-06-04T01:30:09.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12822025-06-04T01:30:09.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
12832025-06-04T01:30:09.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12842025-06-04T01:30:09.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12852025-06-04T01:30:09.090ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
12862025-06-04T01:30:09.090ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.669905ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
12872025-06-04T01:30:09.103ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12882025-06-04T01:30:09.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12892025-06-04T01:30:09.109ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.508263ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12902025-06-04T01:30:15.468ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2202 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 348964f9-0219-475d-969c-f2c8066cfc32 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
12912025-06-04T01:30:19.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
12922025-06-04T01:30:19.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12932025-06-04T01:30:19.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
12942025-06-04T01:30:19.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12952025-06-04T01:30:19.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12962025-06-04T01:30:19.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12972025-06-04T01:30:19.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12982025-06-04T01:30:19.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12992025-06-04T01:30:19.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13002025-06-04T01:30:19.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13012025-06-04T01:30:19.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13022025-06-04T01:30:19.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13032025-06-04T01:30:19.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13042025-06-04T01:30:19.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13052025-06-04T01:30:19.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062025-06-04T01:30:19.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
13072025-06-04T01:30:19.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13082025-06-04T01:30:19.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13092025-06-04T01:30:19.091ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
13102025-06-04T01:30:19.091ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.094555ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
13112025-06-04T01:30:19.101ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.124246ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13122025-06-04T01:30:25.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2116 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = eaadc258-a0cb-40f4-9d6c-176fd6a50ae8 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
13132025-06-04T01:30:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
13142025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13152025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
13162025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
13172025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
13182025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13192025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13202025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
13212025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
13222025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13232025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
13242025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
13252025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
13262025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
13272025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13282025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
13292025-06-04T01:30:29.066ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
13302025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
13312025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13322025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13332025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13342025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
13352025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
13362025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13372025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
13382025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
13392025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13402025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13412025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
13422025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13432025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
13442025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13452025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
13462025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13472025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13482025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13492025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
13502025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13512025-06-04T01:30:29.066ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
13522025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.341\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
13532025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
13542025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
13552025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13562025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
13572025-06-04T01:30:29.066ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:20:29.066761239 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
13582025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
13592025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
13602025-06-04T01:30:29.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13612025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
13622025-06-04T01:30:29.067ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
13632025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
13642025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
13652025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
13662025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
13672025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13682025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13692025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
13702025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
13712025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13722025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
13732025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
13742025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
13752025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13762025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
13772025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13782025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
13792025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
13802025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
13812025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
13822025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
13832025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
13842025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
13852025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
13862025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13872025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
13882025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13892025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
13902025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
13912025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
13922025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13932025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
13942025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13952025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13962025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13972025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13982025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13992025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14002025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14012025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14022025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
14032025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14042025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
14052025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14062025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
14072025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
14082025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14092025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14102025-06-04T01:30:29.067ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 160663233 } })
14112025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14122025-06-04T01:30:29.067ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 160665574 } })
14132025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14142025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
14152025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14162025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
14172025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
14182025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
14192025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
14202025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
14212025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14222025-06-04T01:30:29.067ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
14232025-06-04T01:30:29.068ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
14242025-06-04T01:30:29.068ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.263986ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
14252025-06-04T01:30:29.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14262025-06-04T01:30:29.068ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
14272025-06-04T01:30:29.068ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.147292ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
14282025-06-04T01:30:29.069ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14292025-06-04T01:30:29.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14302025-06-04T01:30:29.069ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14312025-06-04T01:30:29.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14322025-06-04T01:30:29.069ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14332025-06-04T01:30:29.070ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342025-06-04T01:30:29.070ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.665791ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14352025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
14362025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14372025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14382025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
14392025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
14402025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14412025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14422025-06-04T01:30:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14432025-06-04T01:30:29.074ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14442025-06-04T01:30:29.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14452025-06-04T01:30:29.076ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14462025-06-04T01:30:29.076ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14472025-06-04T01:30:29.076ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14482025-06-04T01:30:29.076ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14492025-06-04T01:30:29.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14502025-06-04T01:30:29.077ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14512025-06-04T01:30:29.077ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14522025-06-04T01:30:29.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14532025-06-04T01:30:29.079ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 172369442 } })
14542025-06-04T01:30:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14552025-06-04T01:30:29.079ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14562025-06-04T01:30:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
14572025-06-04T01:30:29.079ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
14582025-06-04T01:30:29.079ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
14592025-06-04T01:30:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14602025-06-04T01:30:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14612025-06-04T01:30:29.080ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14622025-06-04T01:30:29.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14632025-06-04T01:30:29.082ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
14642025-06-04T01:30:29.082ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 15.834415ms iteration = 13 status = {"by_rx":{},"error":null}
14652025-06-04T01:30:29.082ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14662025-06-04T01:30:29.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14672025-06-04T01:30:29.084ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14682025-06-04T01:30:29.084ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
14692025-06-04T01:30:29.084ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14702025-06-04T01:30:29.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14712025-06-04T01:30:29.084ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:20:29.066761239 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
14722025-06-04T01:30:29.084ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.1637ms iteration = 13 status = {"expiration":"2025-06-04T01:20:29.066761239Z","pruned":[]}
14732025-06-04T01:30:29.085ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 178204096 } })
14742025-06-04T01:30:29.085ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
14752025-06-04T01:30:29.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14762025-06-04T01:30:29.087ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
14772025-06-04T01:30:29.087ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
14782025-06-04T01:30:29.087ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 20.263699ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
14792025-06-04T01:30:29.087ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.448054ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
14802025-06-04T01:30:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14812025-06-04T01:30:29.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14822025-06-04T01:30:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14832025-06-04T01:30:29.091ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14842025-06-04T01:30:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
14852025-06-04T01:30:29.095ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
14862025-06-04T01:30:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14872025-06-04T01:30:29.097ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14882025-06-04T01:30:29.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14892025-06-04T01:30:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
14902025-06-04T01:30:29.100ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
14912025-06-04T01:30:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
14922025-06-04T01:30:29.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14932025-06-04T01:30:29.103ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14942025-06-04T01:30:29.103ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14952025-06-04T01:30:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14962025-06-04T01:30:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14972025-06-04T01:30:29.105ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14982025-06-04T01:30:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14992025-06-04T01:30:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15002025-06-04T01:30:29.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15012025-06-04T01:30:29.107ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15022025-06-04T01:30:29.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15032025-06-04T01:30:29.124ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15042025-06-04T01:30:29.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
15052025-06-04T01:30:29.128ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15062025-06-04T01:30:29.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15072025-06-04T01:30:29.129ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
15082025-06-04T01:30:29.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
15092025-06-04T01:30:29.130ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15102025-06-04T01:30:29.131ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 65.079696ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15112025-06-04T01:30:29.132ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15122025-06-04T01:30:29.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15132025-06-04T01:30:29.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15142025-06-04T01:30:29.133ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15152025-06-04T01:30:29.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15162025-06-04T01:30:29.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15172025-06-04T01:30:29.134ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15182025-06-04T01:30:29.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15192025-06-04T01:30:29.135ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15202025-06-04T01:30:29.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15212025-06-04T01:30:29.136ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15222025-06-04T01:30:29.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15232025-06-04T01:30:29.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15242025-06-04T01:30:29.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15252025-06-04T01:30:29.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15262025-06-04T01:30:29.138ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 71.801485ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
15272025-06-04T01:30:29.139ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15282025-06-04T01:30:29.141ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.195483ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
15292025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15302025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15312025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15322025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
15332025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
15342025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
15352025-06-04T01:30:29.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15362025-06-04T01:30:29.143ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15372025-06-04T01:30:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
15382025-06-04T01:30:29.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15392025-06-04T01:30:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
15402025-06-04T01:30:29.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412025-06-04T01:30:29.149ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 82.467975ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
15422025-06-04T01:30:29.149ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15432025-06-04T01:30:29.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15442025-06-04T01:30:29.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 244107166 } })
15452025-06-04T01:30:29.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15462025-06-04T01:30:29.151ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
15472025-06-04T01:30:29.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15482025-06-04T01:30:29.152ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15492025-06-04T01:30:29.152ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15502025-06-04T01:30:29.152ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15512025-06-04T01:30:29.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15522025-06-04T01:30:29.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15532025-06-04T01:30:29.152ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 86.996202ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15542025-06-04T01:30:29.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
15552025-06-04T01:30:29.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15562025-06-04T01:30:29.153ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 86.145398ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
15572025-06-04T01:30:29.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15582025-06-04T01:30:29.155ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15592025-06-04T01:30:29.156ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
15602025-06-04T01:30:29.156ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
15612025-06-04T01:30:29.156ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
15622025-06-04T01:30:29.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15632025-06-04T01:30:29.156ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 90.709157ms iteration = 6 status = {"nsuccess":5}
15642025-06-04T01:30:29.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15652025-06-04T01:30:29.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15662025-06-04T01:30:29.157ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15672025-06-04T01:30:29.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15682025-06-04T01:30:29.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15692025-06-04T01:30:29.161ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
15702025-06-04T01:30:29.161ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 95.155251ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
15712025-06-04T01:30:29.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15722025-06-04T01:30:29.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15732025-06-04T01:30:29.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15742025-06-04T01:30:29.166ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
15752025-06-04T01:30:29.166ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
15762025-06-04T01:30:29.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15772025-06-04T01:30:29.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15782025-06-04T01:30:29.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15792025-06-04T01:30:29.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15802025-06-04T01:30:29.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15812025-06-04T01:30:29.170ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
15822025-06-04T01:30:29.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15832025-06-04T01:30:29.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15842025-06-04T01:30:29.171ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15852025-06-04T01:30:29.171ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
15862025-06-04T01:30:29.171ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15872025-06-04T01:30:29.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15882025-06-04T01:30:29.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15892025-06-04T01:30:29.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15902025-06-04T01:30:29.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15912025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
15922025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 107.146538ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
15932025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
15942025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
15952025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
15962025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 107.94709ms iteration = 14 status = {"generation":1}
15972025-06-04T01:30:29.174ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15982025-06-04T01:30:29.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
15992025-06-04T01:30:29.177ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16002025-06-04T01:30:29.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16012025-06-04T01:30:29.180ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 113.371922ms iteration = 25 status = 0
16022025-06-04T01:30:29.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16032025-06-04T01:30:29.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16042025-06-04T01:30:29.187ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16052025-06-04T01:30:29.188ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16062025-06-04T01:30:29.188ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 122.065517ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
16072025-06-04T01:30:29.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16082025-06-04T01:30:29.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16092025-06-04T01:30:29.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16102025-06-04T01:30:29.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 129.606848ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16112025-06-04T01:30:29.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16122025-06-04T01:30:29.204ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
16132025-06-04T01:30:29.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16142025-06-04T01:30:29.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16152025-06-04T01:30:29.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16162025-06-04T01:30:29.205ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 138.223249ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16172025-06-04T01:30:29.211ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
16182025-06-04T01:30:29.211ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
16192025-06-04T01:30:29.211ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 144.312861ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
16202025-06-04T01:30:29.211ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
16212025-06-04T01:30:29.211ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16222025-06-04T01:30:29.211ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "23d16e4a-b8fd-45e6-acbf-b0f92b28089c", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
16232025-06-04T01:30:29.211ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16242025-06-04T01:30:29.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
16252025-06-04T01:30:29.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
16262025-06-04T01:30:29.214ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16272025-06-04T01:30:29.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16282025-06-04T01:30:29.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16292025-06-04T01:30:29.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16302025-06-04T01:30:29.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16312025-06-04T01:30:29.222ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
16322025-06-04T01:30:29.222ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16332025-06-04T01:30:29.223ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c33138f9-6f7c-48f5-84f1-1cc5f81fece4", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
16342025-06-04T01:30:29.223ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16352025-06-04T01:30:29.225ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16362025-06-04T01:30:29.226ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16372025-06-04T01:30:29.226ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16382025-06-04T01:30:29.226ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 159.124666ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16392025-06-04T01:30:29.246ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
16402025-06-04T01:30:29.246ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 180.137517ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
16412025-06-04T01:30:29.254ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 187.626066ms iteration = 28 status = {"physical_disks_added":0}
16422025-06-04T01:30:29.262ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16432025-06-04T01:30:29.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16442025-06-04T01:30:29.296ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
16452025-06-04T01:30:29.296ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16462025-06-04T01:30:29.297ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a688a92-b280-43c4-bca0-3e973a2a3b73", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
16472025-06-04T01:30:29.297ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16482025-06-04T01:30:29.299ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
16492025-06-04T01:30:29.333ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
16502025-06-04T01:30:29.334ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "678d16b1-b149-4f0d-81c7-460075bd34ba", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16512025-06-04T01:30:29.334ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16522025-06-04T01:30:29.335ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2dc091a2-5683-4298-8d93-1a00478cacd3", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16532025-06-04T01:30:29.335ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16542025-06-04T01:30:29.338ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16552025-06-04T01:30:29.338ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 272.238391ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
16562025-06-04T01:30:29.347ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
16572025-06-04T01:30:29.347ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 281.126651ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
16582025-06-04T01:30:29.377ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16592025-06-04T01:30:29.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16602025-06-04T01:30:29.390ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 323.245983ms iteration = 25 status = {}
16612025-06-04T01:30:29.455ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
16622025-06-04T01:30:29.455ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16632025-06-04T01:30:29.456ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3582ef67-5e60-4b20-917b-ced3a1bd8681", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
16642025-06-04T01:30:29.456ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16652025-06-04T01:30:29.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16662025-06-04T01:30:29.457ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16672025-06-04T01:30:29.458ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 391.89178ms iteration = 25 status = null
16682025-06-04T01:30:29.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16692025-06-04T01:30:29.459ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16702025-06-04T01:30:29.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16712025-06-04T01:30:29.460ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
16722025-06-04T01:30:29.513ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
16732025-06-04T01:30:29.513ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
16742025-06-04T01:30:29.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16752025-06-04T01:30:29.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16762025-06-04T01:30:29.516ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16772025-06-04T01:30:29.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16782025-06-04T01:30:29.518ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16792025-06-04T01:30:29.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16802025-06-04T01:30:29.533ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
16812025-06-04T01:30:29.533ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
16822025-06-04T01:30:29.533ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "83d52988-46b8-4676-aec7-2cfc5cb573ef", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
16832025-06-04T01:30:29.533ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16842025-06-04T01:30:29.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16852025-06-04T01:30:29.539ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16862025-06-04T01:30:29.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16872025-06-04T01:30:29.545ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16882025-06-04T01:30:29.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16892025-06-04T01:30:29.548ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
16902025-06-04T01:30:29.548ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16912025-06-04T01:30:29.549ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e90d5f25-6e6c-49cb-b612-1f5727a83f16", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
16922025-06-04T01:30:29.551ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 485.054345ms iteration = 29 status = {}
16932025-06-04T01:30:29.566ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16942025-06-04T01:30:29.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16952025-06-04T01:30:29.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16962025-06-04T01:30:29.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16972025-06-04T01:30:29.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16982025-06-04T01:30:29.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16992025-06-04T01:30:29.570ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
17002025-06-04T01:30:29.571ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "94282092-eb24-4f98-a5d9-c77b9c91bc99", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17012025-06-04T01:30:29.571ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17022025-06-04T01:30:29.572ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
17032025-06-04T01:30:29.572ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
17042025-06-04T01:30:29.572ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
17052025-06-04T01:30:29.572ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 506.206199ms iteration = 14 status = {"generation":2}
17062025-06-04T01:30:29.573ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "8140d0d7-7624-4805-87d0-73f9e2d1a377", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17072025-06-04T01:30:29.573ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17082025-06-04T01:30:29.575ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1c152c8f-21b0-4e5d-bd7d-61f27fff0157", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17092025-06-04T01:30:29.575ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17102025-06-04T01:30:29.578ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17112025-06-04T01:30:29.578ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 511.676792ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
17122025-06-04T01:30:29.579ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
17132025-06-04T01:30:29.579ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17142025-06-04T01:30:29.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17152025-06-04T01:30:29.591ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
17162025-06-04T01:30:29.591ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
17172025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17182025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17192025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17202025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17212025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17222025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17232025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17242025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17252025-06-04T01:30:29.592ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
17262025-06-04T01:30:29.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17272025-06-04T01:30:29.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17282025-06-04T01:30:29.594ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
17292025-06-04T01:30:29.594ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17302025-06-04T01:30:29.640ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17312025-06-04T01:30:29.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ac491d0d-b671-4f7a-9f2c-ba9e2cc658a7", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
17322025-06-04T01:30:29.693ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
17332025-06-04T01:30:29.693ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17342025-06-04T01:30:29.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cd69f47-ef58-4015-9ef8-0711f7920398", "content-length": "682", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
17352025-06-04T01:30:29.693ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfcae97a-7662-44ed-ab1b-2389189bb350", "content-length": "691", "date": "Wed, 04 Jun 2025 01:30:29 GMT"} })
17362025-06-04T01:30:29.693ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17372025-06-04T01:30:29.693ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17382025-06-04T01:30:29.693ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17392025-06-04T01:30:29.693ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17402025-06-04T01:30:29.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17412025-06-04T01:30:29.695ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 628.815431ms iteration = 27 status = {}
17422025-06-04T01:30:29.698ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17432025-06-04T01:30:29.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17442025-06-04T01:30:29.702ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17452025-06-04T01:30:29.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17462025-06-04T01:30:29.704ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17472025-06-04T01:30:29.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17482025-06-04T01:30:29.711ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 645.091258ms iteration = 25 status = {}
17492025-06-04T01:30:35.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2129 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 8ea95ba2-1ded-4574-92fe-3b0e3ec3836b response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
17502025-06-04T01:30:37.227ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:47425
17512025-06-04T01:30:37.227ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:47425 req_id = 338e7bde-79ef-41db-8769-9e31d9bd4895 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
17522025-06-04T01:30:37.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:47425 req_id = 338e7bde-79ef-41db-8769-9e31d9bd4895 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
17532025-06-04T01:30:37.230ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3243 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:47425 req_id = 338e7bde-79ef-41db-8769-9e31d9bd4895 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
17542025-06-04T01:30:39.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
17552025-06-04T01:30:39.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
17562025-06-04T01:30:39.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17572025-06-04T01:30:39.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17582025-06-04T01:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17592025-06-04T01:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17602025-06-04T01:30:39.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17612025-06-04T01:30:39.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17622025-06-04T01:30:39.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17632025-06-04T01:30:39.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17642025-06-04T01:30:39.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17652025-06-04T01:30:39.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17662025-06-04T01:30:39.105ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17672025-06-04T01:30:39.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17682025-06-04T01:30:39.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17692025-06-04T01:30:39.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
17702025-06-04T01:30:39.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17712025-06-04T01:30:39.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17722025-06-04T01:30:39.120ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
17732025-06-04T01:30:39.120ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.753437ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
17742025-06-04T01:30:39.143ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.688221ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17752025-06-04T01:30:45.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2112 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 720700ff-3f49-45ce-82d6-0ae5ed467640 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
17762025-06-04T01:30:49.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
17772025-06-04T01:30:49.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17782025-06-04T01:30:49.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
17792025-06-04T01:30:49.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17802025-06-04T01:30:49.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17812025-06-04T01:30:49.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17822025-06-04T01:30:49.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17832025-06-04T01:30:49.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17842025-06-04T01:30:49.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17852025-06-04T01:30:49.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17862025-06-04T01:30:49.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17872025-06-04T01:30:49.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17882025-06-04T01:30:49.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17892025-06-04T01:30:49.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17902025-06-04T01:30:49.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17912025-06-04T01:30:49.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
17922025-06-04T01:30:49.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17932025-06-04T01:30:49.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17942025-06-04T01:30:49.089ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
17952025-06-04T01:30:49.089ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.674325ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
17962025-06-04T01:30:49.102ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.644798ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17972025-06-04T01:30:55.452ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:57096
17982025-06-04T01:30:55.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 0c2ccd29-00ae-4129-9d75-ea1d97217d73 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
17992025-06-04T01:30:55.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 0c2ccd29-00ae-4129-9d75-ea1d97217d73 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
18002025-06-04T01:30:55.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3268 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 0c2ccd29-00ae-4129-9d75-ea1d97217d73 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
18012025-06-04T01:30:55.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 79f7ee10-bf87-4b80-9be8-fa51a1ee1572 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
18022025-06-04T01:30:55.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 79f7ee10-bf87-4b80-9be8-fa51a1ee1572 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
18032025-06-04T01:30:55.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2286 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 79f7ee10-bf87-4b80-9be8-fa51a1ee1572 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
18042025-06-04T01:30:55.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2095 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = a0aa1439-41f8-4704-9330-997d5d9181ff response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
18052025-06-04T01:30:56.322ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:53147
18062025-06-04T01:30:56.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53147 req_id = 46a151a0-9f35-45b6-9a98-22bcbbd7370b roles = RoleSet { roles: {} } uri = /metrics/producers
18072025-06-04T01:30:56.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53147 req_id = 46a151a0-9f35-45b6-9a98-22bcbbd7370b resource = Database result = Ok(()) uri = /metrics/producers
18082025-06-04T01:30:56.367ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 4adf0198-0a8a-47c8-a668-dc22afd01e24
18092025-06-04T01:30:56.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 45143 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53147 req_id = 46a151a0-9f35-45b6-9a98-22bcbbd7370b response_code = 201 uri = /metrics/producers
18102025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
18112025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18122025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
18132025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
18142025-06-04T01:30:59.066ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
18152025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18162025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18172025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
18182025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
18192025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18202025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
18212025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18222025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
18232025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18242025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
18252025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
18262025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
18272025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
18282025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18292025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18302025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
18312025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18322025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
18332025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
18342025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
18352025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18362025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18372025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
18382025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18392025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18402025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18412025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18422025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
18432025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
18442025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18452025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18462025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18472025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18482025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18492025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18502025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18512025-06-04T01:30:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18522025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
18532025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18542025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
18552025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
18562025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
18572025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
18582025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
18592025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
18602025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
18612025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18622025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
18632025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
18642025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
18652025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18662025-06-04T01:30:59.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18672025-06-04T01:30:59.069ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18682025-06-04T01:30:59.069ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18692025-06-04T01:30:59.069ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.9051ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
18702025-06-04T01:30:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18712025-06-04T01:30:59.071ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18722025-06-04T01:30:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18732025-06-04T01:30:59.071ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1796, tv_nsec: 164710997 } })
18742025-06-04T01:30:59.071ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18752025-06-04T01:30:59.071ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18762025-06-04T01:30:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18772025-06-04T01:30:59.071ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
18782025-06-04T01:30:59.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18792025-06-04T01:30:59.073ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1796, tv_nsec: 167225907 } })
18802025-06-04T01:30:59.074ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
18812025-06-04T01:30:59.074ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18822025-06-04T01:30:59.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
18832025-06-04T01:30:59.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18842025-06-04T01:30:59.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18852025-06-04T01:30:59.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
18862025-06-04T01:30:59.076ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18872025-06-04T01:30:59.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18882025-06-04T01:30:59.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18892025-06-04T01:30:59.079ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.234824ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
18902025-06-04T01:30:59.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
18912025-06-04T01:30:59.079ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18922025-06-04T01:30:59.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18932025-06-04T01:30:59.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18942025-06-04T01:30:59.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18952025-06-04T01:30:59.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18962025-06-04T01:30:59.083ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1796, tv_nsec: 177181317 } })
18972025-06-04T01:30:59.084ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
18982025-06-04T01:30:59.085ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18992025-06-04T01:30:59.085ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19002025-06-04T01:30:59.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19012025-06-04T01:30:59.111ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19022025-06-04T01:30:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19032025-06-04T01:30:59.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19042025-06-04T01:30:59.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19052025-06-04T01:30:59.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19062025-06-04T01:30:59.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19072025-06-04T01:30:59.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19082025-06-04T01:30:59.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19092025-06-04T01:30:59.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19102025-06-04T01:30:59.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19112025-06-04T01:30:59.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19122025-06-04T01:30:59.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
19132025-06-04T01:30:59.118ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19142025-06-04T01:30:59.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19152025-06-04T01:30:59.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19162025-06-04T01:30:59.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19172025-06-04T01:30:59.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19182025-06-04T01:30:59.121ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19192025-06-04T01:30:59.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19202025-06-04T01:30:59.122ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19212025-06-04T01:30:59.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19222025-06-04T01:30:59.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
19232025-06-04T01:30:59.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19242025-06-04T01:30:59.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19252025-06-04T01:30:59.129ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 62.392231ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
19262025-06-04T01:30:59.133ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
19272025-06-04T01:30:59.133ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 67.531055ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
19282025-06-04T01:30:59.134ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.551126ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
19292025-06-04T01:30:59.138ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19302025-06-04T01:30:59.138ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
19312025-06-04T01:30:59.138ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19322025-06-04T01:30:59.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19332025-06-04T01:30:59.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19342025-06-04T01:30:59.141ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
19352025-06-04T01:30:59.141ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.656334ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
19362025-06-04T01:30:59.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19372025-06-04T01:30:59.141ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19382025-06-04T01:30:59.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19392025-06-04T01:30:59.143ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19402025-06-04T01:30:59.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19412025-06-04T01:30:59.145ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19422025-06-04T01:30:59.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19432025-06-04T01:30:59.147ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19442025-06-04T01:30:59.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19452025-06-04T01:30:59.154ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.557206ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19462025-06-04T01:30:59.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19472025-06-04T01:30:59.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19482025-06-04T01:30:59.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19492025-06-04T01:30:59.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19502025-06-04T01:30:59.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19512025-06-04T01:30:59.166ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 100.547962ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19522025-06-04T01:30:59.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
19532025-06-04T01:30:59.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19542025-06-04T01:30:59.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19552025-06-04T01:30:59.170ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 103.966228ms iteration = 26 status = 0
19562025-06-04T01:30:59.170ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19572025-06-04T01:30:59.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19582025-06-04T01:30:59.172ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 104.791991ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
19592025-06-04T01:30:59.174ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
19602025-06-04T01:30:59.174ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19612025-06-04T01:30:59.175ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "94119c07-57f5-4098-8f47-65eadf73b2aa", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
19622025-06-04T01:30:59.175ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19632025-06-04T01:30:59.176ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19642025-06-04T01:30:59.176ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
19652025-06-04T01:30:59.176ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19662025-06-04T01:30:59.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19672025-06-04T01:30:59.177ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19682025-06-04T01:30:59.179ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 112.596031ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19692025-06-04T01:30:59.184ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
19702025-06-04T01:30:59.184ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 118.830176ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
19712025-06-04T01:30:59.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19722025-06-04T01:30:59.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19732025-06-04T01:30:59.210ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19742025-06-04T01:30:59.210ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
19752025-06-04T01:30:59.210ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 144.023062ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
19762025-06-04T01:30:59.255ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 189.215062ms iteration = 29 status = {"physical_disks_added":0}
19772025-06-04T01:30:59.273ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
19782025-06-04T01:30:59.273ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19792025-06-04T01:30:59.273ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d42cd4a-6046-405c-a892-e9de58aa457f", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
19802025-06-04T01:30:59.273ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19812025-06-04T01:30:59.275ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
19822025-06-04T01:30:59.275ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19832025-06-04T01:30:59.275ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "da06d2eb-95da-47ce-b154-aa732b907a58", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
19842025-06-04T01:30:59.275ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19852025-06-04T01:30:59.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19862025-06-04T01:30:59.276ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19872025-06-04T01:30:59.284ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 218.316668ms iteration = 26 status = {}
19882025-06-04T01:30:59.287ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
19892025-06-04T01:30:59.287ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19902025-06-04T01:30:59.287ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ad09044-18a8-49cc-911a-24640895cfc0", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
19912025-06-04T01:30:59.287ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19922025-06-04T01:30:59.289ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
19932025-06-04T01:30:59.312ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19942025-06-04T01:30:59.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19952025-06-04T01:30:59.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19962025-06-04T01:30:59.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19972025-06-04T01:30:59.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19982025-06-04T01:30:59.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19992025-06-04T01:30:59.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20002025-06-04T01:30:59.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20012025-06-04T01:30:59.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20022025-06-04T01:30:59.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20032025-06-04T01:30:59.322ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
20042025-06-04T01:30:59.322ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
20052025-06-04T01:30:59.322ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "484eadb7-8b7a-4520-b7df-51e869050098", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
20062025-06-04T01:30:59.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20072025-06-04T01:30:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20082025-06-04T01:30:59.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20092025-06-04T01:30:59.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20102025-06-04T01:30:59.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20112025-06-04T01:30:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20122025-06-04T01:30:59.332ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
20132025-06-04T01:30:59.332ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20142025-06-04T01:30:59.332ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e493a3dd-d9f9-4a19-ab8a-d987f2b74464", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
20152025-06-04T01:30:59.334ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 268.499126ms iteration = 30 status = {}
20162025-06-04T01:30:59.344ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20172025-06-04T01:30:59.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20182025-06-04T01:30:59.345ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 279.791824ms iteration = 26 status = null
20192025-06-04T01:30:59.438ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20202025-06-04T01:30:59.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20212025-06-04T01:30:59.453ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
20222025-06-04T01:30:59.453ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a6489db-25c0-4c05-aeec-5a7297fa30a9", "content-length": "691", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
20232025-06-04T01:30:59.453ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
20242025-06-04T01:30:59.453ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
20252025-06-04T01:30:59.453ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
20262025-06-04T01:30:59.455ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 388.930257ms iteration = 28 status = {}
20272025-06-04T01:30:59.455ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
20282025-06-04T01:30:59.455ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20292025-06-04T01:30:59.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20302025-06-04T01:30:59.469ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
20312025-06-04T01:30:59.469ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
20322025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
20332025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
20342025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20352025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20362025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20372025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20382025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20392025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20402025-06-04T01:30:59.470ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
20412025-06-04T01:30:59.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20422025-06-04T01:30:59.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20432025-06-04T01:30:59.472ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
20442025-06-04T01:30:59.472ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20452025-06-04T01:30:59.576ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bfbd8b73-5f7b-4ad9-970b-d02aa8a73ca6", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
20462025-06-04T01:30:59.576ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
20472025-06-04T01:30:59.576ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20482025-06-04T01:30:59.576ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "45039524-6164-4355-a7f3-f81223df7ee9", "content-length": "682", "date": "Wed, 04 Jun 2025 01:30:59 GMT"} })
20492025-06-04T01:30:59.576ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20502025-06-04T01:30:59.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20512025-06-04T01:30:59.580ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20522025-06-04T01:30:59.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20532025-06-04T01:30:59.583ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20542025-06-04T01:30:59.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20552025-06-04T01:30:59.586ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20562025-06-04T01:30:59.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20572025-06-04T01:30:59.594ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 528.121944ms iteration = 26 status = {}
20582025-06-04T01:31:01.074ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:44285
20592025-06-04T01:31:01.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44285 req_id = b3c53aac-fe0d-4053-8e3b-3d96818a3b44 roles = RoleSet { roles: {} } uri = /metrics/producers
20602025-06-04T01:31:01.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44285 req_id = b3c53aac-fe0d-4053-8e3b-3d96818a3b44 resource = Database result = Ok(()) uri = /metrics/producers
20612025-06-04T01:31:01.114ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 0d417b9f-81d7-4f7f-afd4-a2f10841f4be
20622025-06-04T01:31:01.114ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 40275 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44285 req_id = b3c53aac-fe0d-4053-8e3b-3d96818a3b44 response_code = 201 uri = /metrics/producers
20632025-06-04T01:31:04.335ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:64858
20642025-06-04T01:31:04.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64858 req_id = 76e8502d-0a58-4e2b-91ea-386c57f8f73c roles = RoleSet { roles: {} } uri = /metrics/producers
20652025-06-04T01:31:04.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64858 req_id = 76e8502d-0a58-4e2b-91ea-386c57f8f73c resource = Database result = Ok(()) uri = /metrics/producers
20662025-06-04T01:31:04.348ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 5b824dec-17fd-42b4-8865-196e8c0ea94e
20672025-06-04T01:31:04.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 13140 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64858 req_id = 76e8502d-0a58-4e2b-91ea-386c57f8f73c response_code = 201 uri = /metrics/producers
20682025-06-04T01:31:05.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2156 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 7c92aa26-60b0-4471-b8b2-5bf7735f3062 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
20692025-06-04T01:31:06.903ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:45027
20702025-06-04T01:31:06.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45027 req_id = 107745c5-1401-4dd7-a466-cb8525111ccd roles = RoleSet { roles: {} } uri = /metrics/producers
20712025-06-04T01:31:06.903ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45027 req_id = 107745c5-1401-4dd7-a466-cb8525111ccd resource = Database result = Ok(()) uri = /metrics/producers
20722025-06-04T01:31:06.942ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 9ae067c0-87a3-4b45-9e9f-f5ab379cb16d
20732025-06-04T01:31:06.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 39521 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:45027 req_id = 107745c5-1401-4dd7-a466-cb8525111ccd response_code = 201 uri = /metrics/producers
20742025-06-04T01:31:07.230ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:61317
20752025-06-04T01:31:07.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:61317 req_id = 15741889-ebfe-4c33-b9dd-e538886fb4d4 roles = RoleSet { roles: {} } uri = /metrics/producers
20762025-06-04T01:31:07.230ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:61317 req_id = 15741889-ebfe-4c33-b9dd-e538886fb4d4 resource = Database result = Ok(()) uri = /metrics/producers
20772025-06-04T01:31:07.263ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:55343
20782025-06-04T01:31:07.263ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:55343 req_id = a0601c54-b0df-494f-8e94-3725f1cf94f5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
20792025-06-04T01:31:07.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:55343 req_id = a0601c54-b0df-494f-8e94-3725f1cf94f5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
20802025-06-04T01:31:07.266ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2709 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:55343 req_id = a0601c54-b0df-494f-8e94-3725f1cf94f5 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
20812025-06-04T01:31:07.275ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 62a2a719-adba-426d-af2e-301ef11726a2
20822025-06-04T01:31:07.275ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 45221 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:61317 req_id = 15741889-ebfe-4c33-b9dd-e538886fb4d4 response_code = 201 uri = /metrics/producers
20832025-06-04T01:31:09.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
20842025-06-04T01:31:09.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20852025-06-04T01:31:09.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
20862025-06-04T01:31:09.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20872025-06-04T01:31:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20882025-06-04T01:31:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20892025-06-04T01:31:09.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20902025-06-04T01:31:09.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20912025-06-04T01:31:09.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20922025-06-04T01:31:09.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20932025-06-04T01:31:09.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20942025-06-04T01:31:09.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20952025-06-04T01:31:09.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20962025-06-04T01:31:09.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
20972025-06-04T01:31:09.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20982025-06-04T01:31:09.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20992025-06-04T01:31:09.090ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
21002025-06-04T01:31:09.090ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.729614ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
21012025-06-04T01:31:09.098ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21022025-06-04T01:31:09.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21032025-06-04T01:31:09.135ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.719297ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21042025-06-04T01:31:11.781ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:35145
21052025-06-04T01:31:11.781ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35145 req_id = 8fbfc85d-614c-4988-80cc-5bd4317bb0c4 roles = RoleSet { roles: {} } uri = /metrics/producers
21062025-06-04T01:31:11.781ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35145 req_id = 8fbfc85d-614c-4988-80cc-5bd4317bb0c4 resource = Database result = Ok(()) uri = /metrics/producers
21072025-06-04T01:31:11.791ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 0387bc2c-d5fa-45d7-848d-f11bcc323d46
21082025-06-04T01:31:11.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 9777 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35145 req_id = 8fbfc85d-614c-4988-80cc-5bd4317bb0c4 response_code = 201 uri = /metrics/producers
21092025-06-04T01:31:15.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2510 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = de526259-a3b9-4f86-8e98-b97e3c14d684 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
21102025-06-04T01:31:19.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
21112025-06-04T01:31:19.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21122025-06-04T01:31:19.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
21132025-06-04T01:31:19.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21142025-06-04T01:31:19.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21152025-06-04T01:31:19.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21162025-06-04T01:31:19.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21172025-06-04T01:31:19.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21182025-06-04T01:31:19.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21192025-06-04T01:31:19.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21202025-06-04T01:31:19.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21212025-06-04T01:31:19.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21222025-06-04T01:31:19.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21232025-06-04T01:31:19.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
21242025-06-04T01:31:19.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21252025-06-04T01:31:19.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21262025-06-04T01:31:19.097ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
21272025-06-04T01:31:19.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.569466ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
21282025-06-04T01:31:19.102ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21292025-06-04T01:31:19.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21302025-06-04T01:31:19.136ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.19938ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21312025-06-04T01:31:23.138ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 5
21322025-06-04T01:31:23.138ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
21332025-06-04T01:31:23.168ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
21342025-06-04T01:31:23.168ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:43988
21352025-06-04T01:31:23.168ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43988 req_id = 1c0eb95a-4344-4217-8202-a5f2c872d68b roles = RoleSet { roles: {} } uri = /metrics/producers
21362025-06-04T01:31:23.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43988 req_id = 1c0eb95a-4344-4217-8202-a5f2c872d68b resource = Database result = Ok(()) uri = /metrics/producers
21372025-06-04T01:31:23.181ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 34942533-fcec-49f1-bc91-3bae11367eac
21382025-06-04T01:31:23.181ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 13196 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43988 req_id = 1c0eb95a-4344-4217-8202-a5f2c872d68b response_code = 201 uri = /metrics/producers
21392025-06-04T01:31:23.181ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "1c0eb95a-4344-4217-8202-a5f2c872d68b", "content-length": "41", "date": "Wed, 04 Jun 2025 01:31:23 GMT"} })
21402025-06-04T01:31:23.184ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
21412025-06-04T01:31:25.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = ed875783-9894-49d4-9397-39525dc83619 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
21422025-06-04T01:31:25.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = ed875783-9894-49d4-9397-39525dc83619 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
21432025-06-04T01:31:25.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3608 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = ed875783-9894-49d4-9397-39525dc83619 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
21442025-06-04T01:31:25.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 025219e7-6c38-44d8-bce2-5cf28d7c7010 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
21452025-06-04T01:31:25.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 025219e7-6c38-44d8-bce2-5cf28d7c7010 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
21462025-06-04T01:31:25.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2060 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 025219e7-6c38-44d8-bce2-5cf28d7c7010 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
21472025-06-04T01:31:25.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2224 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = deec4bb9-dba4-438c-b9ff-a4e9f944507c response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
21482025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
21492025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21502025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
21512025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
21522025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
21532025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21542025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21552025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
21562025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
21572025-06-04T01:31:29.065ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
21582025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.881\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
21592025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
21602025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21612025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
21622025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
21632025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21642025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21652025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
21662025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21672025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
21682025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
21692025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21702025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
21712025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
21722025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
21732025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21742025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21752025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
21762025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
21772025-06-04T01:31:29.065ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:21:29.065538867 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
21782025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
21792025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
21802025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21812025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
21822025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21832025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
21842025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21852025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
21862025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
21872025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
21882025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
21892025-06-04T01:31:29.065ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
21902025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21912025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
21922025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
21932025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
21942025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
21952025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
21962025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21972025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
21982025-06-04T01:31:29.065ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
21992025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22002025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22012025-06-04T01:31:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22022025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22032025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22042025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22052025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22062025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22072025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22082025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22092025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22102025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
22112025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
22122025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22132025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22142025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
22152025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22162025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
22172025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
22182025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22192025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
22202025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
22212025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
22222025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
22232025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
22242025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
22252025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
22262025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
22272025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
22282025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
22292025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
22302025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
22312025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22322025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22332025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
22342025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
22352025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22362025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
22372025-06-04T01:31:29.066ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1826, tv_nsec: 160082297 } })
22382025-06-04T01:31:29.066ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1826, tv_nsec: 160109508 } })
22392025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
22402025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
22412025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
22422025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22432025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22442025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
22452025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
22462025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
22472025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
22482025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
22492025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.159033ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
22502025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
22512025-06-04T01:31:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.168613ms iteration = 15 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
22522025-06-04T01:31:29.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22532025-06-04T01:31:29.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22542025-06-04T01:31:29.068ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22552025-06-04T01:31:29.069ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22562025-06-04T01:31:29.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22572025-06-04T01:31:29.070ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.701232ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
22582025-06-04T01:31:29.070ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22592025-06-04T01:31:29.070ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1826, tv_nsec: 164521862 } })
22602025-06-04T01:31:29.070ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
22612025-06-04T01:31:29.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22622025-06-04T01:31:29.071ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22632025-06-04T01:31:29.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22642025-06-04T01:31:29.071ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22652025-06-04T01:31:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22662025-06-04T01:31:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22672025-06-04T01:31:29.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22682025-06-04T01:31:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22692025-06-04T01:31:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22702025-06-04T01:31:29.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22712025-06-04T01:31:29.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22722025-06-04T01:31:29.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22732025-06-04T01:31:29.073ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22742025-06-04T01:31:29.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22752025-06-04T01:31:29.075ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22762025-06-04T01:31:29.076ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22772025-06-04T01:31:29.076ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:21:29.065538867 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
22782025-06-04T01:31:29.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22792025-06-04T01:31:29.076ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 11.146213ms iteration = 14 status = {"expiration":"2025-06-04T01:21:29.065538867Z","pruned":[]}
22802025-06-04T01:31:29.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22812025-06-04T01:31:29.077ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22822025-06-04T01:31:29.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
22832025-06-04T01:31:29.078ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22842025-06-04T01:31:29.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
22852025-06-04T01:31:29.078ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22862025-06-04T01:31:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22872025-06-04T01:31:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22882025-06-04T01:31:29.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22892025-06-04T01:31:29.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22902025-06-04T01:31:29.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
22912025-06-04T01:31:29.080ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
22922025-06-04T01:31:29.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
22932025-06-04T01:31:29.082ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
22942025-06-04T01:31:29.082ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 16.648998ms iteration = 14 status = {"by_rx":{},"error":null}
22952025-06-04T01:31:29.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1826, tv_nsec: 176675153 } })
22962025-06-04T01:31:29.083ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
22972025-06-04T01:31:29.084ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 18.097488ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
22982025-06-04T01:31:29.085ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.023824ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
22992025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23002025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23012025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23022025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 21.221976ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
23032025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23042025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23052025-06-04T01:31:29.087ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1826, tv_nsec: 181482788 } })
23062025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23072025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23082025-06-04T01:31:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23092025-06-04T01:31:29.087ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
23102025-06-04T01:31:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23112025-06-04T01:31:29.090ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23122025-06-04T01:31:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23132025-06-04T01:31:29.092ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
23142025-06-04T01:31:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23152025-06-04T01:31:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
23162025-06-04T01:31:29.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23172025-06-04T01:31:29.093ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
23182025-06-04T01:31:29.093ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
23192025-06-04T01:31:29.093ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
23202025-06-04T01:31:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23212025-06-04T01:31:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23222025-06-04T01:31:29.094ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
23232025-06-04T01:31:29.094ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 29.218561ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
23242025-06-04T01:31:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
23252025-06-04T01:31:29.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23262025-06-04T01:31:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23272025-06-04T01:31:29.102ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23282025-06-04T01:31:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23292025-06-04T01:31:29.104ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23302025-06-04T01:31:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23312025-06-04T01:31:29.107ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23322025-06-04T01:31:29.111ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 45.471786ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
23332025-06-04T01:31:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23342025-06-04T01:31:29.112ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23352025-06-04T01:31:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23362025-06-04T01:31:29.125ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23372025-06-04T01:31:29.125ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
23382025-06-04T01:31:29.125ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23392025-06-04T01:31:29.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23402025-06-04T01:31:29.127ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23412025-06-04T01:31:29.127ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23422025-06-04T01:31:29.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23432025-06-04T01:31:29.129ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23442025-06-04T01:31:29.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23452025-06-04T01:31:29.130ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23462025-06-04T01:31:29.130ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23472025-06-04T01:31:29.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23482025-06-04T01:31:29.132ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23492025-06-04T01:31:29.132ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
23502025-06-04T01:31:29.132ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.014731ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
23512025-06-04T01:31:29.133ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23522025-06-04T01:31:29.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23532025-06-04T01:31:29.133ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23542025-06-04T01:31:29.134ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23552025-06-04T01:31:29.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23562025-06-04T01:31:29.134ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23572025-06-04T01:31:29.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23582025-06-04T01:31:29.135ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23592025-06-04T01:31:29.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23602025-06-04T01:31:29.136ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23612025-06-04T01:31:29.136ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 71.2619ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23622025-06-04T01:31:29.137ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 71.352763ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
23632025-06-04T01:31:29.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23642025-06-04T01:31:29.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23652025-06-04T01:31:29.138ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
23662025-06-04T01:31:29.138ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
23672025-06-04T01:31:29.138ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 72.521936ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
23682025-06-04T01:31:29.138ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23692025-06-04T01:31:29.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23702025-06-04T01:31:29.139ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.005268ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23712025-06-04T01:31:29.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
23722025-06-04T01:31:29.140ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23732025-06-04T01:31:29.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23742025-06-04T01:31:29.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23752025-06-04T01:31:29.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23762025-06-04T01:31:29.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23772025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23782025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23792025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
23802025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23812025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23822025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23832025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
23842025-06-04T01:31:29.146ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
23852025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 81.179889ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
23862025-06-04T01:31:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23872025-06-04T01:31:29.148ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
23882025-06-04T01:31:29.148ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 83.314769ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
23892025-06-04T01:31:29.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23902025-06-04T01:31:29.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23912025-06-04T01:31:29.166ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
23922025-06-04T01:31:29.166ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
23932025-06-04T01:31:29.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23942025-06-04T01:31:29.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23952025-06-04T01:31:29.172ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23962025-06-04T01:31:29.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23972025-06-04T01:31:29.174ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23982025-06-04T01:31:29.174ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
23992025-06-04T01:31:29.174ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24002025-06-04T01:31:29.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24012025-06-04T01:31:29.182ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
24022025-06-04T01:31:29.182ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
24032025-06-04T01:31:29.182ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
24042025-06-04T01:31:29.182ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24052025-06-04T01:31:29.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24062025-06-04T01:31:29.210ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
24072025-06-04T01:31:29.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
24082025-06-04T01:31:29.213ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
24092025-06-04T01:31:29.216ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
24102025-06-04T01:31:29.221ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
24112025-06-04T01:31:29.221ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24122025-06-04T01:31:29.221ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2fb8e72-e602-45bb-9237-86512e02b0db", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
24132025-06-04T01:31:29.221ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24142025-06-04T01:31:29.223ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24152025-06-04T01:31:29.223ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
24162025-06-04T01:31:29.223ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24172025-06-04T01:31:29.224ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "90b48c27-5514-49c3-8b7c-ef342286ea58", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
24182025-06-04T01:31:29.224ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24192025-06-04T01:31:29.225ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24202025-06-04T01:31:29.250ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
24212025-06-04T01:31:29.250ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24222025-06-04T01:31:29.250ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6cb6de-9027-4aa9-a59f-f3adb0534800", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
24232025-06-04T01:31:29.250ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24242025-06-04T01:31:29.253ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
24252025-06-04T01:31:29.300ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
24262025-06-04T01:31:29.300ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24272025-06-04T01:31:29.301ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "86bc3977-994b-4474-bf53-ea033cea542c", "content-length": "372", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
24282025-06-04T01:31:29.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24292025-06-04T01:31:29.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24302025-06-04T01:31:29.304ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 238.896555ms iteration = 27 status = 0
24312025-06-04T01:31:29.304ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24322025-06-04T01:31:29.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
24332025-06-04T01:31:29.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24342025-06-04T01:31:29.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24352025-06-04T01:31:29.314ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
24362025-06-04T01:31:29.314ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
24372025-06-04T01:31:29.314ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
24382025-06-04T01:31:29.314ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 249.138873ms iteration = 15 status = {"generation":1}
24392025-06-04T01:31:29.343ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24402025-06-04T01:31:29.344ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24412025-06-04T01:31:29.344ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
24422025-06-04T01:31:29.344ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 279.035642ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
24432025-06-04T01:31:29.346ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 280.698529ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
24442025-06-04T01:31:29.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24452025-06-04T01:31:29.347ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24462025-06-04T01:31:29.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24472025-06-04T01:31:29.349ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24482025-06-04T01:31:29.349ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24492025-06-04T01:31:29.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 283.24897ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24502025-06-04T01:31:29.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24512025-06-04T01:31:29.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24522025-06-04T01:31:29.350ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24532025-06-04T01:31:29.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24542025-06-04T01:31:29.352ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24552025-06-04T01:31:29.352ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
24562025-06-04T01:31:29.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24572025-06-04T01:31:29.354ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24582025-06-04T01:31:29.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24592025-06-04T01:31:29.363ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
24602025-06-04T01:31:29.363ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
24612025-06-04T01:31:29.363ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0472ced8-6563-4bd8-8c62-3ae9cbe35224", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
24622025-06-04T01:31:29.363ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24632025-06-04T01:31:29.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24642025-06-04T01:31:29.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24652025-06-04T01:31:29.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24662025-06-04T01:31:29.366ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
24672025-06-04T01:31:29.367ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f4ef00e1-2e65-4cf6-81bd-46fb64757cfb", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24682025-06-04T01:31:29.367ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24692025-06-04T01:31:29.369ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "5be7f5d7-511a-4ec7-a98b-8835e2fee6d0", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24702025-06-04T01:31:29.369ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24712025-06-04T01:31:29.370ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24722025-06-04T01:31:29.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24732025-06-04T01:31:29.371ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24742025-06-04T01:31:29.371ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 305.978259ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
24752025-06-04T01:31:29.373ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24762025-06-04T01:31:29.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24772025-06-04T01:31:29.373ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
24782025-06-04T01:31:29.373ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 308.202611ms iteration = 27 status = {}
24792025-06-04T01:31:29.373ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24802025-06-04T01:31:29.374ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "72009137-d6cd-48cf-8a46-11c9dc3f25d9", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
24812025-06-04T01:31:29.375ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 310.263619ms iteration = 27 status = null
24822025-06-04T01:31:29.376ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 310.845525ms iteration = 31 status = {}
24832025-06-04T01:31:29.393ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
24842025-06-04T01:31:29.393ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
24852025-06-04T01:31:29.393ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24862025-06-04T01:31:29.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24872025-06-04T01:31:29.394ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24882025-06-04T01:31:29.395ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24892025-06-04T01:31:29.398ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
24902025-06-04T01:31:29.398ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24912025-06-04T01:31:29.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24922025-06-04T01:31:29.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24932025-06-04T01:31:29.401ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24942025-06-04T01:31:29.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24952025-06-04T01:31:29.403ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
24962025-06-04T01:31:29.404ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
24972025-06-04T01:31:29.404ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
24982025-06-04T01:31:29.404ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 338.510432ms iteration = 15 status = {"generation":2}
24992025-06-04T01:31:29.451ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
25002025-06-04T01:31:29.452ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "af0ff5e3-a71b-4174-bb23-200cd882eea4", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25012025-06-04T01:31:29.452ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25022025-06-04T01:31:29.453ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 388.167435ms iteration = 30 status = {"physical_disks_added":0}
25032025-06-04T01:31:29.455ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "9b9f6a58-ef26-44df-b173-eb0738cc3ebc", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25042025-06-04T01:31:29.455ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25052025-06-04T01:31:29.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3ed717a5-5a60-46c1-859e-40c6a3ff5b75", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25062025-06-04T01:31:29.457ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25072025-06-04T01:31:29.460ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
25082025-06-04T01:31:29.460ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 394.608936ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
25092025-06-04T01:31:29.471ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) }
25102025-06-04T01:31:29.471ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 405.976126ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
25112025-06-04T01:31:29.478ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
25122025-06-04T01:31:29.478ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 413.156326ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
25132025-06-04T01:31:29.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25142025-06-04T01:31:29.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25152025-06-04T01:31:29.493ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
25162025-06-04T01:31:29.493ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fbe08d0-2395-4d4c-808c-bf456552346d", "content-length": "691", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
25172025-06-04T01:31:29.494ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
25182025-06-04T01:31:29.494ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
25192025-06-04T01:31:29.494ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
25202025-06-04T01:31:29.494ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
25212025-06-04T01:31:29.494ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25222025-06-04T01:31:29.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25232025-06-04T01:31:29.495ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 429.636609ms iteration = 29 status = {}
25242025-06-04T01:31:29.513ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
25252025-06-04T01:31:29.513ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
25262025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
25272025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
25282025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25292025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25302025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25312025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
25322025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
25332025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
25342025-06-04T01:31:29.514ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
25352025-06-04T01:31:29.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25362025-06-04T01:31:29.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25372025-06-04T01:31:29.520ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
25382025-06-04T01:31:29.520ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
25392025-06-04T01:31:29.622ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8496130b-ccbd-406d-ae6d-8d5dd24d147c", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
25402025-06-04T01:31:29.623ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
25412025-06-04T01:31:29.623ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
25422025-06-04T01:31:29.623ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "64699a6a-ae69-40a0-b50d-d50c9ea3f1cd", "content-length": "682", "date": "Wed, 04 Jun 2025 01:31:29 GMT"} })
25432025-06-04T01:31:29.623ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25442025-06-04T01:31:29.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25452025-06-04T01:31:29.627ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25462025-06-04T01:31:29.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25472025-06-04T01:31:29.630ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25482025-06-04T01:31:29.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25492025-06-04T01:31:29.633ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25502025-06-04T01:31:29.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25512025-06-04T01:31:29.639ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 574.305771ms iteration = 27 status = {}
25522025-06-04T01:31:29.950ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:60392
25532025-06-04T01:31:29.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60392 req_id = ca1070dc-d2e9-4e68-858e-9bdce6b2f53a roles = RoleSet { roles: {} } uri = /metrics/producers
25542025-06-04T01:31:29.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60392 req_id = ca1070dc-d2e9-4e68-858e-9bdce6b2f53a resource = Database result = Ok(()) uri = /metrics/producers
25552025-06-04T01:31:29.998ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 62a2a719-adba-426d-af2e-301ef11726a2
25562025-06-04T01:31:29.998ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 48316 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60392 req_id = ca1070dc-d2e9-4e68-858e-9bdce6b2f53a response_code = 201 uri = /metrics/producers
25572025-06-04T01:31:35.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 64961473-be69-4c3a-bd50-c95a6ca8b88c response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
25582025-06-04T01:31:37.298ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:32816
25592025-06-04T01:31:37.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:32816 req_id = ccecdf80-ee86-430a-9589-a7ae24f0c9b1 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
25602025-06-04T01:31:37.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:32816 req_id = ccecdf80-ee86-430a-9589-a7ae24f0c9b1 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
25612025-06-04T01:31:37.301ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3497 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:32816 req_id = ccecdf80-ee86-430a-9589-a7ae24f0c9b1 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
25622025-06-04T01:31:39.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
25632025-06-04T01:31:39.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25642025-06-04T01:31:39.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
25652025-06-04T01:31:39.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25662025-06-04T01:31:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25672025-06-04T01:31:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25682025-06-04T01:31:39.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25692025-06-04T01:31:39.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25702025-06-04T01:31:39.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25712025-06-04T01:31:39.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25722025-06-04T01:31:39.104ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25732025-06-04T01:31:39.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25742025-06-04T01:31:39.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25752025-06-04T01:31:39.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25762025-06-04T01:31:39.140ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.770709ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25772025-06-04T01:31:39.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25782025-06-04T01:31:39.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
25792025-06-04T01:31:39.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25802025-06-04T01:31:39.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25812025-06-04T01:31:39.165ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
25822025-06-04T01:31:39.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 99.512273ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
25832025-06-04T01:31:40.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 31c7754f-f952-4900-af8e-69f2306fc9e8 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
25842025-06-04T01:31:40.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 31c7754f-f952-4900-af8e-69f2306fc9e8 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
25852025-06-04T01:31:40.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2553 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 31c7754f-f952-4900-af8e-69f2306fc9e8 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
25862025-06-04T01:31:40.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 482e7266-da4a-4e3a-b621-43d9354993bb roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
25872025-06-04T01:31:40.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 482e7266-da4a-4e3a-b621-43d9354993bb resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
25882025-06-04T01:31:40.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 482e7266-da4a-4e3a-b621-43d9354993bb response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
25892025-06-04T01:31:45.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2233 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 56d452a2-2174-4959-86d4-cc54a000b1f7 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
25902025-06-04T01:31:49.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
25912025-06-04T01:31:49.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
25922025-06-04T01:31:49.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25932025-06-04T01:31:49.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25942025-06-04T01:31:49.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25952025-06-04T01:31:49.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25962025-06-04T01:31:49.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25972025-06-04T01:31:49.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25982025-06-04T01:31:49.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25992025-06-04T01:31:49.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26002025-06-04T01:31:49.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26012025-06-04T01:31:49.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26022025-06-04T01:31:49.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26032025-06-04T01:31:49.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26042025-06-04T01:31:49.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26052025-06-04T01:31:49.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
26062025-06-04T01:31:49.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26072025-06-04T01:31:49.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26082025-06-04T01:31:49.089ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
26092025-06-04T01:31:49.089ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.523308ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
26102025-06-04T01:31:49.098ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.77221ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26112025-06-04T01:31:55.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2173 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = b61001b0-35f9-4482-a148-58b9da577f9d response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
26122025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
26132025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
26142025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26152025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26162025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
26172025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
26182025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
26192025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
26202025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26212025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
26222025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26232025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
26242025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
26252025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26262025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
26272025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26282025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
26292025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26302025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
26312025-06-04T01:31:59.065ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
26322025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
26332025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26342025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
26352025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26362025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
26372025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26382025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26392025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
26402025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26412025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
26422025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26432025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
26442025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26452025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26462025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26472025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26482025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26492025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26502025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26512025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26522025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26532025-06-04T01:31:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26542025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
26552025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26562025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
26572025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
26582025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
26592025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
26602025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
26612025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26622025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
26632025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
26642025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
26652025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26662025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
26672025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
26682025-06-04T01:31:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26692025-06-04T01:31:59.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26702025-06-04T01:31:59.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26712025-06-04T01:31:59.068ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1856, tv_nsec: 162898916 } })
26722025-06-04T01:31:59.069ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
26732025-06-04T01:31:59.069ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.552058ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
26742025-06-04T01:31:59.069ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26752025-06-04T01:31:59.070ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26762025-06-04T01:31:59.071ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
26772025-06-04T01:31:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
26782025-06-04T01:31:59.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26792025-06-04T01:31:59.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26802025-06-04T01:31:59.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26812025-06-04T01:31:59.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26822025-06-04T01:31:59.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26832025-06-04T01:31:59.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26842025-06-04T01:31:59.074ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1856, tv_nsec: 168329359 } })
26852025-06-04T01:31:59.074ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26862025-06-04T01:31:59.074ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
26872025-06-04T01:31:59.074ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26882025-06-04T01:31:59.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26892025-06-04T01:31:59.078ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26902025-06-04T01:31:59.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26912025-06-04T01:31:59.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26922025-06-04T01:31:59.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26932025-06-04T01:31:59.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26942025-06-04T01:31:59.082ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 17.614485ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26952025-06-04T01:31:59.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26962025-06-04T01:31:59.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26972025-06-04T01:31:59.084ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26982025-06-04T01:31:59.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26992025-06-04T01:31:59.085ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1856, tv_nsec: 179649056 } })
27002025-06-04T01:31:59.085ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
27012025-06-04T01:31:59.108ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27022025-06-04T01:31:59.108ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
27032025-06-04T01:31:59.108ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27042025-06-04T01:31:59.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27052025-06-04T01:31:59.110ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 44.262562ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
27062025-06-04T01:31:59.111ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27072025-06-04T01:31:59.112ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27082025-06-04T01:31:59.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27092025-06-04T01:31:59.114ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27102025-06-04T01:31:59.114ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27112025-06-04T01:31:59.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27122025-06-04T01:31:59.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27132025-06-04T01:31:59.116ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 49.789037ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
27142025-06-04T01:31:59.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27152025-06-04T01:31:59.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27162025-06-04T01:31:59.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27172025-06-04T01:31:59.118ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.228686ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
27182025-06-04T01:31:59.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27192025-06-04T01:31:59.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27202025-06-04T01:31:59.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27212025-06-04T01:31:59.119ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27222025-06-04T01:31:59.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27232025-06-04T01:31:59.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27242025-06-04T01:31:59.121ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
27252025-06-04T01:31:59.121ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27262025-06-04T01:31:59.121ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27272025-06-04T01:31:59.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27282025-06-04T01:31:59.121ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27292025-06-04T01:31:59.121ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d400371f-fa3d-489e-a0e8-ff08f5376d32", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
27302025-06-04T01:31:59.121ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27312025-06-04T01:31:59.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27322025-06-04T01:31:59.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27332025-06-04T01:31:59.123ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27342025-06-04T01:31:59.123ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27352025-06-04T01:31:59.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
27362025-06-04T01:31:59.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27372025-06-04T01:31:59.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27382025-06-04T01:31:59.125ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27392025-06-04T01:31:59.125ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27402025-06-04T01:31:59.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27412025-06-04T01:31:59.130ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27422025-06-04T01:31:59.131ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
27432025-06-04T01:31:59.131ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.300181ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
27442025-06-04T01:31:59.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27452025-06-04T01:31:59.132ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27462025-06-04T01:31:59.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27472025-06-04T01:31:59.132ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27482025-06-04T01:31:59.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
27492025-06-04T01:31:59.138ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 73.256046ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27502025-06-04T01:31:59.138ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
27512025-06-04T01:31:59.138ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27522025-06-04T01:31:59.138ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
27532025-06-04T01:31:59.138ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 73.438292ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
27542025-06-04T01:31:59.138ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb5db19-d7b4-43cd-b12a-e0211ee28934", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
27552025-06-04T01:31:59.138ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27562025-06-04T01:31:59.139ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27572025-06-04T01:31:59.145ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
27582025-06-04T01:31:59.145ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27592025-06-04T01:31:59.145ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bae6b0aa-289e-44cc-9818-15941522eecc", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
27602025-06-04T01:31:59.145ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27612025-06-04T01:31:59.148ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
27622025-06-04T01:31:59.149ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27632025-06-04T01:31:59.149ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27642025-06-04T01:31:59.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27652025-06-04T01:31:59.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27662025-06-04T01:31:59.160ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27672025-06-04T01:31:59.160ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
27682025-06-04T01:31:59.160ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27692025-06-04T01:31:59.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27702025-06-04T01:31:59.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27712025-06-04T01:31:59.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
27722025-06-04T01:31:59.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27732025-06-04T01:31:59.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27742025-06-04T01:31:59.176ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 109.941346ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
27752025-06-04T01:31:59.177ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 112.518779ms iteration = 28 status = 0
27762025-06-04T01:31:59.185ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27772025-06-04T01:31:59.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27782025-06-04T01:31:59.187ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27792025-06-04T01:31:59.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27802025-06-04T01:31:59.188ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 122.253832ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
27812025-06-04T01:31:59.196ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27822025-06-04T01:31:59.196ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
27832025-06-04T01:31:59.196ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 131.144161ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
27842025-06-04T01:31:59.196ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 131.755799ms iteration = 28 status = {}
27852025-06-04T01:31:59.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27862025-06-04T01:31:59.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27872025-06-04T01:31:59.225ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
27882025-06-04T01:31:59.225ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 160.144715ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
27892025-06-04T01:31:59.245ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 180.515406ms iteration = 31 status = {"physical_disks_added":0}
27902025-06-04T01:31:59.294ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
27912025-06-04T01:31:59.294ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27922025-06-04T01:31:59.295ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbddfd0d-89ae-4759-86ac-68805cdb45c6", "content-length": "372", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
27932025-06-04T01:31:59.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27942025-06-04T01:31:59.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27952025-06-04T01:31:59.304ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27962025-06-04T01:31:59.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27972025-06-04T01:31:59.305ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27982025-06-04T01:31:59.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
27992025-06-04T01:31:59.307ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28002025-06-04T01:31:59.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28012025-06-04T01:31:59.309ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28022025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28032025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28042025-06-04T01:31:59.310ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
28052025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
28062025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c3578006-0b80-4d07-804d-2525fab940d3", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
28072025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28082025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28092025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28102025-06-04T01:31:59.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28112025-06-04T01:31:59.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28122025-06-04T01:31:59.311ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 246.705564ms iteration = 28 status = null
28132025-06-04T01:31:59.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28142025-06-04T01:31:59.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28152025-06-04T01:31:59.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28162025-06-04T01:31:59.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28172025-06-04T01:31:59.318ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
28182025-06-04T01:31:59.319ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28192025-06-04T01:31:59.319ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9fa43bdc-cbe5-4fef-b420-9d2d56fa55bb", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
28202025-06-04T01:31:59.321ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 255.922073ms iteration = 32 status = {}
28212025-06-04T01:31:59.375ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
28222025-06-04T01:31:59.376ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1613afb-fabe-491b-a884-7cc75f1df5d4", "content-length": "691", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
28232025-06-04T01:31:59.376ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
28242025-06-04T01:31:59.376ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
28252025-06-04T01:31:59.376ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
28262025-06-04T01:31:59.377ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 312.515892ms iteration = 30 status = {}
28272025-06-04T01:31:59.409ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28282025-06-04T01:31:59.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28292025-06-04T01:31:59.412ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
28302025-06-04T01:31:59.412ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28312025-06-04T01:31:59.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28322025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
28332025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
28342025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
28352025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
28362025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
28372025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
28382025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
28392025-06-04T01:31:59.431ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
28402025-06-04T01:31:59.432ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
28412025-06-04T01:31:59.432ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
28422025-06-04T01:31:59.432ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
28432025-06-04T01:31:59.432ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28442025-06-04T01:31:59.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28452025-06-04T01:31:59.435ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
28462025-06-04T01:31:59.435ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
28472025-06-04T01:31:59.538ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "00e500c0-4701-4124-a69c-17ea0927bdf2", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
28482025-06-04T01:31:59.538ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
28492025-06-04T01:31:59.538ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
28502025-06-04T01:31:59.538ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3a09ddb-4052-4294-b7a3-f0b98215ae67", "content-length": "682", "date": "Wed, 04 Jun 2025 01:31:59 GMT"} })
28512025-06-04T01:31:59.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28522025-06-04T01:31:59.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28532025-06-04T01:31:59.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28542025-06-04T01:31:59.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28552025-06-04T01:31:59.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28562025-06-04T01:31:59.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28572025-06-04T01:31:59.547ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28582025-06-04T01:31:59.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28592025-06-04T01:31:59.555ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 489.85528ms iteration = 28 status = {}
28602025-06-04T01:32:05.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 190f1f3f-7e86-40d0-80d4-7b44c93b66cb response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
28612025-06-04T01:32:07.332ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:41984
28622025-06-04T01:32:07.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41984 req_id = 3b52e552-fc66-4458-b0e6-a350f77f7666 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
28632025-06-04T01:32:07.332ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41984 req_id = 3b52e552-fc66-4458-b0e6-a350f77f7666 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
28642025-06-04T01:32:07.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3099 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:41984 req_id = 3b52e552-fc66-4458-b0e6-a350f77f7666 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
28652025-06-04T01:32:09.064ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
28662025-06-04T01:32:09.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28672025-06-04T01:32:09.064ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
28682025-06-04T01:32:09.064ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28692025-06-04T01:32:09.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28702025-06-04T01:32:09.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28712025-06-04T01:32:09.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28722025-06-04T01:32:09.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28732025-06-04T01:32:09.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28742025-06-04T01:32:09.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28752025-06-04T01:32:09.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28762025-06-04T01:32:09.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28772025-06-04T01:32:09.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28782025-06-04T01:32:09.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28792025-06-04T01:32:09.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28802025-06-04T01:32:09.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
28812025-06-04T01:32:09.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28822025-06-04T01:32:09.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28832025-06-04T01:32:09.086ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
28842025-06-04T01:32:09.086ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.910395ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
28852025-06-04T01:32:09.099ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.072855ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28862025-06-04T01:32:10.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = be30fed4-c267-4ca5-8ca6-6d9ea9d72428 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
28872025-06-04T01:32:10.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = be30fed4-c267-4ca5-8ca6-6d9ea9d72428 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
28882025-06-04T01:32:10.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2274 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = be30fed4-c267-4ca5-8ca6-6d9ea9d72428 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
28892025-06-04T01:32:10.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 1f5ebd39-bba5-4d4f-b36f-f2a95fa3d730 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
28902025-06-04T01:32:10.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 1f5ebd39-bba5-4d4f-b36f-f2a95fa3d730 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
28912025-06-04T01:32:10.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2179 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57096 req_id = 1f5ebd39-bba5-4d4f-b36f-f2a95fa3d730 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
28922025-06-04T01:32:15.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2159 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = f534617d-c334-40d7-a183-c587859d5183 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
28932025-06-04T01:32:19.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
28942025-06-04T01:32:19.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28952025-06-04T01:32:19.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
28962025-06-04T01:32:19.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28972025-06-04T01:32:19.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28982025-06-04T01:32:19.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28992025-06-04T01:32:19.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29002025-06-04T01:32:19.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29012025-06-04T01:32:19.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29022025-06-04T01:32:19.069ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.466946ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29032025-06-04T01:32:19.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29042025-06-04T01:32:19.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29052025-06-04T01:32:19.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29062025-06-04T01:32:19.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29072025-06-04T01:32:19.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29082025-06-04T01:32:19.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29092025-06-04T01:32:19.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
29102025-06-04T01:32:19.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29112025-06-04T01:32:19.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29122025-06-04T01:32:19.086ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
29132025-06-04T01:32:19.086ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.704879ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
29142025-06-04T01:32:25.467ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2792 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 740827d5-1f8a-46bd-8f1f-f5a6dfb3ee18 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
29152025-06-04T01:32:29.064ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
29162025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
29172025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29182025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Timeout
29192025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
29202025-06-04T01:32:29.065ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
29212025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.391\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
29222025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
29232025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29242025-06-04T01:32:29.065ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
29252025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
29262025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
29272025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
29282025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29292025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29302025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29312025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
29322025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29332025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
29342025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
29352025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29362025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
29372025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29382025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
29392025-06-04T01:32:29.065ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:22:29.065132355 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
29402025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
29412025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
29422025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29432025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
29442025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
29452025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
29462025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29472025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29482025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
29492025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
29502025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29512025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
29522025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
29532025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29542025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
29552025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
29562025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
29572025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
29582025-06-04T01:32:29.065ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
29592025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
29602025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
29612025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
29622025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
29632025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29642025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29652025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
29662025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29672025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29682025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29692025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29702025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
29712025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29722025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29732025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29742025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29752025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
29762025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
29772025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29782025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
29792025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29802025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 29 reason = Timeout
29812025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
29822025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
29832025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
29842025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29852025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
29862025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
29872025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29882025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
29892025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
29902025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
29912025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 15 reason = Timeout
29922025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
29932025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
29942025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
29952025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29962025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29972025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 15 reason = Timeout
29982025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
29992025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
30002025-06-04T01:32:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30012025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30022025-06-04T01:32:29.066ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1886, tv_nsec: 160074097 } })
30032025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30042025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30052025-06-04T01:32:29.066ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1886, tv_nsec: 160109708 } })
30062025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
30072025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
30082025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
30092025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
30102025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
30112025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
30122025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
30132025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30142025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30152025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
30162025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.245875ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
30172025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
30182025-06-04T01:32:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.332267ms iteration = 16 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
30192025-06-04T01:32:29.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30202025-06-04T01:32:29.067ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30212025-06-04T01:32:29.068ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.955261ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
30222025-06-04T01:32:29.069ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30232025-06-04T01:32:29.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30242025-06-04T01:32:29.069ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30252025-06-04T01:32:29.069ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:22:29.065132355 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
30262025-06-04T01:32:29.069ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.836986ms iteration = 15 status = {"expiration":"2025-06-04T01:22:29.065132355Z","pruned":[]}
30272025-06-04T01:32:29.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30282025-06-04T01:32:29.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30292025-06-04T01:32:29.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30302025-06-04T01:32:29.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30312025-06-04T01:32:29.070ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30322025-06-04T01:32:29.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30332025-06-04T01:32:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30342025-06-04T01:32:29.072ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30352025-06-04T01:32:29.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30362025-06-04T01:32:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30372025-06-04T01:32:29.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30382025-06-04T01:32:29.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30392025-06-04T01:32:29.073ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1886, tv_nsec: 167761713 } })
30402025-06-04T01:32:29.073ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
30412025-06-04T01:32:29.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30422025-06-04T01:32:29.074ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30432025-06-04T01:32:29.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30442025-06-04T01:32:29.074ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30452025-06-04T01:32:29.074ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30462025-06-04T01:32:29.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30472025-06-04T01:32:29.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30482025-06-04T01:32:29.075ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30492025-06-04T01:32:29.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30502025-06-04T01:32:29.075ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30512025-06-04T01:32:29.076ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1886, tv_nsec: 170581562 } })
30522025-06-04T01:32:29.076ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
30532025-06-04T01:32:29.077ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30542025-06-04T01:32:29.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30552025-06-04T01:32:29.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
30562025-06-04T01:32:29.078ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30572025-06-04T01:32:29.078ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30582025-06-04T01:32:29.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
30592025-06-04T01:32:29.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30602025-06-04T01:32:29.078ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30612025-06-04T01:32:29.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30622025-06-04T01:32:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
30632025-06-04T01:32:29.079ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
30642025-06-04T01:32:29.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
30652025-06-04T01:32:29.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30662025-06-04T01:32:29.080ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30672025-06-04T01:32:29.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30682025-06-04T01:32:29.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30692025-06-04T01:32:29.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30702025-06-04T01:32:29.082ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
30712025-06-04T01:32:29.082ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 16.499473ms iteration = 15 status = {"by_rx":{},"error":null}
30722025-06-04T01:32:29.083ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30732025-06-04T01:32:29.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30742025-06-04T01:32:29.084ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.589422ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
30752025-06-04T01:32:29.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30762025-06-04T01:32:29.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30772025-06-04T01:32:29.087ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30782025-06-04T01:32:29.088ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30792025-06-04T01:32:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30802025-06-04T01:32:29.088ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1886, tv_nsec: 182849106 } })
30812025-06-04T01:32:29.088ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
30822025-06-04T01:32:29.089ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30832025-06-04T01:32:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30842025-06-04T01:32:29.089ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30852025-06-04T01:32:29.089ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30862025-06-04T01:32:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30872025-06-04T01:32:29.090ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30882025-06-04T01:32:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30892025-06-04T01:32:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30902025-06-04T01:32:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30912025-06-04T01:32:29.098ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
30922025-06-04T01:32:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
30932025-06-04T01:32:29.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30942025-06-04T01:32:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
30952025-06-04T01:32:29.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30962025-06-04T01:32:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30972025-06-04T01:32:29.129ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30982025-06-04T01:32:29.129ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30992025-06-04T01:32:29.129ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
31002025-06-04T01:32:29.129ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31012025-06-04T01:32:29.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31022025-06-04T01:32:29.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31032025-06-04T01:32:29.131ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.017694ms iteration = 29 status = {"errors":[],"finish_invoked_ok":[]}
31042025-06-04T01:32:29.131ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31052025-06-04T01:32:29.132ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
31062025-06-04T01:32:29.132ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
31072025-06-04T01:32:29.132ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 66.504267ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
31082025-06-04T01:32:29.133ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.303707ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31092025-06-04T01:32:29.133ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31102025-06-04T01:32:29.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31112025-06-04T01:32:29.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31122025-06-04T01:32:29.135ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31132025-06-04T01:32:29.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31142025-06-04T01:32:29.139ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31152025-06-04T01:32:29.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31162025-06-04T01:32:29.140ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31172025-06-04T01:32:29.141ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.017124ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
31182025-06-04T01:32:29.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
31192025-06-04T01:32:29.144ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31202025-06-04T01:32:29.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31212025-06-04T01:32:29.145ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 79.984445ms iteration = 15 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
31222025-06-04T01:32:29.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31232025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
31242025-06-04T01:32:29.146ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
31252025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.817388ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
31262025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
31272025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
31282025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
31292025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31302025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31312025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 80.677214ms iteration = 29 status = {"errors":[],"requests_created_ok":[]}
31322025-06-04T01:32:29.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31332025-06-04T01:32:29.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31342025-06-04T01:32:29.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31352025-06-04T01:32:29.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31362025-06-04T01:32:29.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31372025-06-04T01:32:29.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31382025-06-04T01:32:29.153ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 88.644048ms iteration = 32 status = {"physical_disks_added":0}
31392025-06-04T01:32:29.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31402025-06-04T01:32:29.154ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31412025-06-04T01:32:29.154ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
31422025-06-04T01:32:29.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31432025-06-04T01:32:29.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31442025-06-04T01:32:29.154ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 89.550844ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
31452025-06-04T01:32:29.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31462025-06-04T01:32:29.156ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31472025-06-04T01:32:29.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31482025-06-04T01:32:29.158ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
31492025-06-04T01:32:29.158ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 92.670981ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
31502025-06-04T01:32:29.158ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
31512025-06-04T01:32:29.158ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.848134ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
31522025-06-04T01:32:29.159ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31532025-06-04T01:32:29.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31542025-06-04T01:32:29.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31552025-06-04T01:32:29.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31562025-06-04T01:32:29.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31572025-06-04T01:32:29.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31582025-06-04T01:32:29.161ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
31592025-06-04T01:32:29.161ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31602025-06-04T01:32:29.161ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
31612025-06-04T01:32:29.161ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 96.142559ms iteration = 16 status = {"generation":2}
31622025-06-04T01:32:29.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
31632025-06-04T01:32:29.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31642025-06-04T01:32:29.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31652025-06-04T01:32:29.164ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 99.863013ms iteration = 29 status = 0
31662025-06-04T01:32:29.181ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31672025-06-04T01:32:29.181ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
31682025-06-04T01:32:29.181ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31692025-06-04T01:32:29.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31702025-06-04T01:32:29.188ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31712025-06-04T01:32:29.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31722025-06-04T01:32:29.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31732025-06-04T01:32:29.190ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 125.074151ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
31742025-06-04T01:32:29.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31752025-06-04T01:32:29.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31762025-06-04T01:32:29.193ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31772025-06-04T01:32:29.193ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31782025-06-04T01:32:29.193ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31792025-06-04T01:32:29.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31802025-06-04T01:32:29.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
31812025-06-04T01:32:29.196ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31822025-06-04T01:32:29.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31832025-06-04T01:32:29.198ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
31842025-06-04T01:32:29.198ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
31852025-06-04T01:32:29.198ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
31862025-06-04T01:32:29.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31872025-06-04T01:32:29.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31882025-06-04T01:32:29.200ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31892025-06-04T01:32:29.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31902025-06-04T01:32:29.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31912025-06-04T01:32:29.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31922025-06-04T01:32:29.217ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
31932025-06-04T01:32:29.221ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
31942025-06-04T01:32:29.224ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31952025-06-04T01:32:29.224ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
31962025-06-04T01:32:29.224ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 159.195619ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
31972025-06-04T01:32:29.230ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
31982025-06-04T01:32:29.230ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 165.854945ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
31992025-06-04T01:32:29.234ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 168.653974ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
32002025-06-04T01:32:29.237ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) }
32012025-06-04T01:32:29.237ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 171.992618ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
32022025-06-04T01:32:29.238ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
32032025-06-04T01:32:29.238ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
32042025-06-04T01:32:29.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32052025-06-04T01:32:29.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32062025-06-04T01:32:29.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32072025-06-04T01:32:29.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32082025-06-04T01:32:29.243ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32092025-06-04T01:32:29.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32102025-06-04T01:32:29.245ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
32112025-06-04T01:32:29.245ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32122025-06-04T01:32:29.246ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
32132025-06-04T01:32:29.246ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
32142025-06-04T01:32:29.246ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
32152025-06-04T01:32:29.246ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 180.834356ms iteration = 16 status = {"generation":1}
32162025-06-04T01:32:29.246ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0182c040-b501-4cad-9cc2-d0d0c1d47d7a", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
32172025-06-04T01:32:29.246ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32182025-06-04T01:32:29.250ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32192025-06-04T01:32:29.259ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
32202025-06-04T01:32:29.259ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 194.470608ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
32212025-06-04T01:32:29.265ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32222025-06-04T01:32:29.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32232025-06-04T01:32:29.267ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32242025-06-04T01:32:29.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32252025-06-04T01:32:29.269ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32262025-06-04T01:32:29.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32272025-06-04T01:32:29.271ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32282025-06-04T01:32:29.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32292025-06-04T01:32:29.273ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32302025-06-04T01:32:29.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32312025-06-04T01:32:29.312ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32322025-06-04T01:32:29.312ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32332025-06-04T01:32:29.313ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "366c0967-fe55-4164-8f61-717e1723eded", "content-length": "372", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
32342025-06-04T01:32:29.313ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32352025-06-04T01:32:29.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32362025-06-04T01:32:29.356ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
32372025-06-04T01:32:29.356ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32382025-06-04T01:32:29.357ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "57edac32-2721-409a-847f-ac6a993509cd", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
32392025-06-04T01:32:29.357ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32402025-06-04T01:32:29.359ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32412025-06-04T01:32:29.361ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
32422025-06-04T01:32:29.361ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32432025-06-04T01:32:29.362ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "130d6c54-d81e-46a9-931c-b2edd245d334", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
32442025-06-04T01:32:29.362ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32452025-06-04T01:32:29.363ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
32462025-06-04T01:32:29.363ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
32472025-06-04T01:32:29.364ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "997eab05-c46d-458e-99ea-2472fb806c25", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
32482025-06-04T01:32:29.364ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32492025-06-04T01:32:29.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32502025-06-04T01:32:29.364ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
32512025-06-04T01:32:29.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32522025-06-04T01:32:29.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32532025-06-04T01:32:29.370ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32542025-06-04T01:32:29.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32552025-06-04T01:32:29.374ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
32562025-06-04T01:32:29.374ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32572025-06-04T01:32:29.375ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "93f35fc7-f266-4308-96f7-1308903f3baf", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
32582025-06-04T01:32:29.376ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 311.929936ms iteration = 33 status = {}
32592025-06-04T01:32:29.379ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
32602025-06-04T01:32:29.379ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2f462e5a-4770-476e-9f31-7c351cf92ed2", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32612025-06-04T01:32:29.379ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32622025-06-04T01:32:29.381ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f96339fd-0a7b-40e2-acbe-f7e0f9a30bc2", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32632025-06-04T01:32:29.381ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32642025-06-04T01:32:29.383ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
32652025-06-04T01:32:29.383ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 318.034418ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
32662025-06-04T01:32:29.387ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
32672025-06-04T01:32:29.405ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 340.027194ms iteration = 29 status = {}
32682025-06-04T01:32:29.542ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
32692025-06-04T01:32:29.542ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ff4b5ed5-04ce-4a6f-8ac3-f3f2da762161", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32702025-06-04T01:32:29.542ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32712025-06-04T01:32:29.544ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "d7461f8f-235b-4f0a-987c-2a5bdb3c5bbf", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32722025-06-04T01:32:29.544ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32732025-06-04T01:32:29.546ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3673eec9-6322-43f4-b170-b56e191c5516", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32742025-06-04T01:32:29.546ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32752025-06-04T01:32:29.548ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32762025-06-04T01:32:29.548ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 482.972937ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
32772025-06-04T01:32:29.571ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32782025-06-04T01:32:29.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32792025-06-04T01:32:29.573ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32802025-06-04T01:32:29.574ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5805f2d-6836-4482-8dc1-4769b632cb57", "content-length": "691", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
32812025-06-04T01:32:29.574ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
32822025-06-04T01:32:29.574ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
32832025-06-04T01:32:29.574ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
32842025-06-04T01:32:29.575ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 510.52465ms iteration = 29 status = null
32852025-06-04T01:32:29.576ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 511.103066ms iteration = 31 status = {}
32862025-06-04T01:32:29.618ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32872025-06-04T01:32:29.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32882025-06-04T01:32:29.621ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
32892025-06-04T01:32:29.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32902025-06-04T01:32:29.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32912025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
32922025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
32932025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
32942025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
32952025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32962025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32972025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32982025-06-04T01:32:29.631ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32992025-06-04T01:32:29.632ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33002025-06-04T01:32:29.632ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33012025-06-04T01:32:29.632ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
33022025-06-04T01:32:29.632ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33032025-06-04T01:32:29.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33042025-06-04T01:32:29.634ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
33052025-06-04T01:32:29.634ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33062025-06-04T01:32:29.735ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5d8a9705-c0cf-4a81-bd7c-fd2a710d7b8a", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
33072025-06-04T01:32:29.735ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
33082025-06-04T01:32:29.735ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33092025-06-04T01:32:29.736ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "acc179df-7388-4cca-bd6a-bf29986aecf7", "content-length": "682", "date": "Wed, 04 Jun 2025 01:32:29 GMT"} })
33102025-06-04T01:32:29.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33112025-06-04T01:32:29.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33122025-06-04T01:32:29.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33132025-06-04T01:32:29.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33142025-06-04T01:32:29.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33152025-06-04T01:32:29.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33162025-06-04T01:32:29.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33172025-06-04T01:32:29.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33182025-06-04T01:32:29.753ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 688.701071ms iteration = 29 status = {}
33192025-06-04T01:32:35.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2151 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = b04a3ee5-e7d1-4cf1-832f-22c4ed57372d response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
33202025-06-04T01:32:37.367ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:39551
33212025-06-04T01:32:37.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:39551 req_id = 504c99b0-3bf3-44ce-81e6-76d773ca3dec roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
33222025-06-04T01:32:37.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:39551 req_id = 504c99b0-3bf3-44ce-81e6-76d773ca3dec resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
33232025-06-04T01:32:37.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3556 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:39551 req_id = 504c99b0-3bf3-44ce-81e6-76d773ca3dec response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
33242025-06-04T01:32:39.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
33252025-06-04T01:32:39.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
33262025-06-04T01:32:39.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33272025-06-04T01:32:39.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33282025-06-04T01:32:39.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33292025-06-04T01:32:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33302025-06-04T01:32:39.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33312025-06-04T01:32:39.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33322025-06-04T01:32:39.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33332025-06-04T01:32:39.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33342025-06-04T01:32:39.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33352025-06-04T01:32:39.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33362025-06-04T01:32:39.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33372025-06-04T01:32:39.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
33382025-06-04T01:32:39.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33392025-06-04T01:32:39.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33402025-06-04T01:32:39.088ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
33412025-06-04T01:32:39.088ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.905203ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
33422025-06-04T01:32:39.097ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33432025-06-04T01:32:39.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33442025-06-04T01:32:39.129ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.796341ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33452025-06-04T01:32:45.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2236 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = dd6aeb64-8553-4eca-a909-f17082e2e500 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
33462025-06-04T01:32:49.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
33472025-06-04T01:32:49.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33482025-06-04T01:32:49.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
33492025-06-04T01:32:49.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33502025-06-04T01:32:49.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33512025-06-04T01:32:49.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33522025-06-04T01:32:49.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33532025-06-04T01:32:49.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33542025-06-04T01:32:49.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33552025-06-04T01:32:49.070ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.182165ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33562025-06-04T01:32:49.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33572025-06-04T01:32:49.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33582025-06-04T01:32:49.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33592025-06-04T01:32:49.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33602025-06-04T01:32:49.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33612025-06-04T01:32:49.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33622025-06-04T01:32:49.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
33632025-06-04T01:32:49.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33642025-06-04T01:32:49.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33652025-06-04T01:32:49.088ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
33662025-06-04T01:32:49.088ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.449828ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
33672025-06-04T01:32:55.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2193 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 382cbe47-8cbc-4009-b356-3012a349c78f response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
33682025-06-04T01:32:59.064ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
33692025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
33702025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33712025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
33722025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33732025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33742025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
33752025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
33762025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33772025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
33782025-06-04T01:32:59.065ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
33792025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33802025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
33812025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
33822025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33832025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33842025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
33852025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
33862025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33872025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
33882025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
33892025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
33902025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33912025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
33922025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
33932025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33942025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33952025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
33962025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33972025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33982025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33992025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34002025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34012025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34022025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
34032025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34042025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34052025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34062025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34072025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34082025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34092025-06-04T01:32:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34102025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
34112025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34122025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
34132025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
34142025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 30 reason = Timeout
34152025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
34162025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
34172025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34182025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
34192025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
34202025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
34212025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
34222025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
34232025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34242025-06-04T01:32:59.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
34252025-06-04T01:32:59.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34262025-06-04T01:32:59.068ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34272025-06-04T01:32:59.069ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34282025-06-04T01:32:59.069ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.301281ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
34292025-06-04T01:32:59.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34302025-06-04T01:32:59.070ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34312025-06-04T01:32:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34322025-06-04T01:32:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34332025-06-04T01:32:59.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34342025-06-04T01:32:59.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34352025-06-04T01:32:59.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34362025-06-04T01:32:59.072ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34372025-06-04T01:32:59.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34382025-06-04T01:32:59.073ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34392025-06-04T01:32:59.073ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1916, tv_nsec: 168460722 } })
34402025-06-04T01:32:59.074ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
34412025-06-04T01:32:59.074ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34422025-06-04T01:32:59.074ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34432025-06-04T01:32:59.074ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
34442025-06-04T01:32:59.074ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34452025-06-04T01:32:59.075ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1916, tv_nsec: 169504571 } })
34462025-06-04T01:32:59.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34472025-06-04T01:32:59.075ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
34482025-06-04T01:32:59.075ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34492025-06-04T01:32:59.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34502025-06-04T01:32:59.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34512025-06-04T01:32:59.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34522025-06-04T01:32:59.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
34532025-06-04T01:32:59.076ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34542025-06-04T01:32:59.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34552025-06-04T01:32:59.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34562025-06-04T01:32:59.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34572025-06-04T01:32:59.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34582025-06-04T01:32:59.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34592025-06-04T01:32:59.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34602025-06-04T01:32:59.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34612025-06-04T01:32:59.083ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34622025-06-04T01:32:59.083ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34632025-06-04T01:32:59.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34642025-06-04T01:32:59.085ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34652025-06-04T01:32:59.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34662025-06-04T01:32:59.089ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
34672025-06-04T01:32:59.089ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.590605ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
34682025-06-04T01:32:59.106ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34692025-06-04T01:32:59.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34702025-06-04T01:32:59.109ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34712025-06-04T01:32:59.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34722025-06-04T01:32:59.113ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34732025-06-04T01:32:59.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34742025-06-04T01:32:59.116ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34752025-06-04T01:32:59.119ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34762025-06-04T01:32:59.119ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34772025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34782025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34792025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34802025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.168311ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
34812025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34822025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34832025-06-04T01:32:59.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34842025-06-04T01:32:59.122ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.614049ms iteration = 30 status = {"errors":[],"finish_invoked_ok":[]}
34852025-06-04T01:32:59.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
34862025-06-04T01:32:59.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34872025-06-04T01:32:59.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34882025-06-04T01:32:59.124ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 58.687838ms iteration = 30 status = {"errors":[],"requests_created_ok":[]}
34892025-06-04T01:32:59.124ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34902025-06-04T01:32:59.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34912025-06-04T01:32:59.125ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34922025-06-04T01:32:59.125ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
34932025-06-04T01:32:59.125ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34942025-06-04T01:32:59.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34952025-06-04T01:32:59.126ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34962025-06-04T01:32:59.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34972025-06-04T01:32:59.127ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34982025-06-04T01:32:59.127ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
34992025-06-04T01:32:59.127ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 62.747501ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
35002025-06-04T01:32:59.127ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.665379ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35012025-06-04T01:32:59.131ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35022025-06-04T01:32:59.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35032025-06-04T01:32:59.135ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35042025-06-04T01:32:59.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35052025-06-04T01:32:59.136ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
35062025-06-04T01:32:59.136ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35072025-06-04T01:32:59.136ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "33256ae4-5c07-4470-a159-0a41587c0841", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35082025-06-04T01:32:59.136ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35092025-06-04T01:32:59.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35102025-06-04T01:32:59.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35112025-06-04T01:32:59.138ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35122025-06-04T01:32:59.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
35132025-06-04T01:32:59.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35142025-06-04T01:32:59.140ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
35152025-06-04T01:32:59.140ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.662323ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
35162025-06-04T01:32:59.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35172025-06-04T01:32:59.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
35182025-06-04T01:32:59.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35192025-06-04T01:32:59.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35202025-06-04T01:32:59.141ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35212025-06-04T01:32:59.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35222025-06-04T01:32:59.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1916, tv_nsec: 237799979 } })
35232025-06-04T01:32:59.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
35242025-06-04T01:32:59.145ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 80.122469ms iteration = 30 status = 0
35252025-06-04T01:32:59.159ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
35262025-06-04T01:32:59.159ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35272025-06-04T01:32:59.160ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "563d0c86-58c8-4f92-a6d5-67d2e7940238", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35282025-06-04T01:32:59.160ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35292025-06-04T01:32:59.162ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35302025-06-04T01:32:59.165ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 100.970264ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35312025-06-04T01:32:59.168ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
35322025-06-04T01:32:59.168ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 103.783713ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
35332025-06-04T01:32:59.171ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 105.467361ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
35342025-06-04T01:32:59.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35352025-06-04T01:32:59.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35362025-06-04T01:32:59.176ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35372025-06-04T01:32:59.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35382025-06-04T01:32:59.187ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 121.702447ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35392025-06-04T01:32:59.221ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 155.994679ms iteration = 30 status = {}
35402025-06-04T01:32:59.228ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35412025-06-04T01:32:59.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35422025-06-04T01:32:59.230ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35432025-06-04T01:32:59.231ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
35442025-06-04T01:32:59.231ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35452025-06-04T01:32:59.231ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b95dc456-0541-4438-8bcf-29eb524f911d", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35462025-06-04T01:32:59.231ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35472025-06-04T01:32:59.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35482025-06-04T01:32:59.232ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35492025-06-04T01:32:59.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35502025-06-04T01:32:59.232ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
35512025-06-04T01:32:59.234ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35522025-06-04T01:32:59.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35532025-06-04T01:32:59.236ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35542025-06-04T01:32:59.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35552025-06-04T01:32:59.240ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35562025-06-04T01:32:59.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35572025-06-04T01:32:59.247ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
35582025-06-04T01:32:59.247ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
35592025-06-04T01:32:59.248ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce813970-ef4c-4a8d-be31-d6a83906e455", "content-length": "372", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35602025-06-04T01:32:59.248ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35612025-06-04T01:32:59.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35622025-06-04T01:32:59.250ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 184.959982ms iteration = 30 status = null
35632025-06-04T01:32:59.276ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 211.882357ms iteration = 33 status = {"physical_disks_added":0}
35642025-06-04T01:32:59.294ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
35652025-06-04T01:32:59.294ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
35662025-06-04T01:32:59.295ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "90354cab-d785-4490-8081-f00b8e1e0d6b", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35672025-06-04T01:32:59.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35682025-06-04T01:32:59.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35692025-06-04T01:32:59.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35702025-06-04T01:32:59.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35712025-06-04T01:32:59.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35722025-06-04T01:32:59.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35732025-06-04T01:32:59.304ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
35742025-06-04T01:32:59.304ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
35752025-06-04T01:32:59.304ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9b6944f1-7d37-4038-9491-2d6a1b386d08", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35762025-06-04T01:32:59.307ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 242.043444ms iteration = 34 status = {}
35772025-06-04T01:32:59.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35782025-06-04T01:32:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35792025-06-04T01:32:59.335ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
35802025-06-04T01:32:59.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35812025-06-04T01:32:59.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35822025-06-04T01:32:59.337ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
35832025-06-04T01:32:59.337ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "01a94567-27ed-430a-bcad-7451f0ee4c2e", "content-length": "691", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
35842025-06-04T01:32:59.337ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
35852025-06-04T01:32:59.337ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
35862025-06-04T01:32:59.338ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
35872025-06-04T01:32:59.340ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 275.070631ms iteration = 32 status = {}
35882025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
35892025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
35902025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
35912025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
35922025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35932025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35942025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35952025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
35962025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
35972025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
35982025-06-04T01:32:59.348ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
35992025-06-04T01:32:59.348ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36002025-06-04T01:32:59.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36012025-06-04T01:32:59.350ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
36022025-06-04T01:32:59.350ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36032025-06-04T01:32:59.470ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5e67d083-8af6-4f2f-bc11-0d2183955a90", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
36042025-06-04T01:32:59.470ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
36052025-06-04T01:32:59.470ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
36062025-06-04T01:32:59.476ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac4103a9-8cc1-4ea3-abb2-76694bb9a7d4", "content-length": "682", "date": "Wed, 04 Jun 2025 01:32:59 GMT"} })
36072025-06-04T01:32:59.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36082025-06-04T01:32:59.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36092025-06-04T01:32:59.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36102025-06-04T01:32:59.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36112025-06-04T01:32:59.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36122025-06-04T01:32:59.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36132025-06-04T01:32:59.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36142025-06-04T01:32:59.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36152025-06-04T01:32:59.494ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 428.948621ms iteration = 30 status = {}
36162025-06-04T01:33:05.466ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 39aacaae-9504-4dc3-8a7f-b34d18614c10 response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
36172025-06-04T01:33:07.403ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:59537
36182025-06-04T01:33:07.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:59537 req_id = d636c3ac-8d80-4df8-91af-7b08038b0847 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
36192025-06-04T01:33:07.403ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:59537 req_id = d636c3ac-8d80-4df8-91af-7b08038b0847 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
36202025-06-04T01:33:07.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3574 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:59537 req_id = d636c3ac-8d80-4df8-91af-7b08038b0847 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
36212025-06-04T01:33:09.064ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
36222025-06-04T01:33:09.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
36232025-06-04T01:33:09.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36242025-06-04T01:33:09.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36252025-06-04T01:33:09.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36262025-06-04T01:33:09.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36272025-06-04T01:33:09.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36282025-06-04T01:33:09.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36292025-06-04T01:33:09.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36302025-06-04T01:33:09.070ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.229657ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36312025-06-04T01:33:09.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36322025-06-04T01:33:09.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36332025-06-04T01:33:09.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36342025-06-04T01:33:09.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36352025-06-04T01:33:09.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36362025-06-04T01:33:09.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36372025-06-04T01:33:09.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
36382025-06-04T01:33:09.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36392025-06-04T01:33:09.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36402025-06-04T01:33:09.087ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
36412025-06-04T01:33:09.087ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.345308ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
36422025-06-04T01:33:15.465ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2219 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = 10e46d51-e36d-4a3b-b4c0-d7a2000a3e6b response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
36432025-06-04T01:33:19.064ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
36442025-06-04T01:33:19.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36452025-06-04T01:33:19.064ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
36462025-06-04T01:33:19.064ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36472025-06-04T01:33:19.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36482025-06-04T01:33:19.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36492025-06-04T01:33:19.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36502025-06-04T01:33:19.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36512025-06-04T01:33:19.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36522025-06-04T01:33:19.069ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36532025-06-04T01:33:19.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36542025-06-04T01:33:19.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36552025-06-04T01:33:19.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36562025-06-04T01:33:19.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36572025-06-04T01:33:19.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36582025-06-04T01:33:19.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
36592025-06-04T01:33:19.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36602025-06-04T01:33:19.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36612025-06-04T01:33:19.087ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
36622025-06-04T01:33:19.087ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.088468ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
36632025-06-04T01:33:19.100ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.708082ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36642025-06-04T01:33:25.451ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:55541
36652025-06-04T01:33:25.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55541 req_id = 876ee5af-43e4-48a7-a3b7-111926c07233 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
36662025-06-04T01:33:25.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55541 req_id = 876ee5af-43e4-48a7-a3b7-111926c07233 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
36672025-06-04T01:33:25.465ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2228 local_addr = [fd00:1122:3344:101::c]:39922 method = GET remote_addr = [fd00:1122:3344:101::d]:40271 req_id = b89552c6-a052-4176-aa3a-c174962d105d response_code = 200 uri = /34942533-fcec-49f1-bc91-3bae11367eac
36682025-06-04T01:33:25.541ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 90179 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55541 req_id = 876ee5af-43e4-48a7-a3b7-111926c07233 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
36692025-06-04T01:33:25.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55541 req_id = 11f3d79e-a654-4c9f-8b56-97f8b8948a5e roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
36702025-06-04T01:33:25.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55541 req_id = 11f3d79e-a654-4c9f-8b56-97f8b8948a5e resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
36712025-06-04T01:33:25.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2410 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55541 req_id = 11f3d79e-a654-4c9f-8b56-97f8b8948a5e response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
36722025-06-04T01:33:26.400ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:52344
36732025-06-04T01:33:26.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52344 req_id = 906b0e0e-8ca2-49c1-bd7d-1206c305cf64 roles = RoleSet { roles: {} } uri = /metrics/producers
36742025-06-04T01:33:26.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52344 req_id = 906b0e0e-8ca2-49c1-bd7d-1206c305cf64 resource = Database result = Ok(()) uri = /metrics/producers
36752025-06-04T01:33:26.437ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 4adf0198-0a8a-47c8-a668-dc22afd01e24
36762025-06-04T01:33:26.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 37480 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52344 req_id = 906b0e0e-8ca2-49c1-bd7d-1206c305cf64 response_code = 201 uri = /metrics/producers
36772025-06-04T01:33:29.064ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
36782025-06-04T01:33:29.064ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
36792025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36802025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
36812025-06-04T01:33:29.065ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:23:29.064954156 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
36822025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
36832025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
36842025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 7 reason = Timeout
36852025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
36862025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36872025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
36882025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36892025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Timeout
36902025-06-04T01:33:29.065ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
36912025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36922025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
36932025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
36942025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
36952025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36962025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36972025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
36982025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36992025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 17 reason = Timeout
37002025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 33 reason = Timeout
37012025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37022025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37032025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
37042025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37052025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
37062025-06-04T01:33:29.065ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
37072025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37082025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
37092025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37102025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
37112025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37122025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
37132025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37142025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
37152025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37162025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
37172025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
37182025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 17 reason = Timeout
37192025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
37202025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37212025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
37222025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 17 reason = Timeout
37232025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
37242025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
37252025-06-04T01:33:29.065ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
37262025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.62\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
37272025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 17 reason = Timeout
37282025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
37292025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37302025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
37312025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37322025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
37332025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 16 reason = Timeout
37342025-06-04T01:33:29.065ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
37352025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
37362025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
37372025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
37382025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37392025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
37402025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
37412025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
37422025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 31 reason = Timeout
37432025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
37442025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 16 reason = Timeout
37452025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
37462025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
37472025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37482025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 17 reason = Timeout
37492025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
37502025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37512025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
37522025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
37532025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
37542025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 4 reason = Timeout
37552025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
37562025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
37572025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
37582025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37592025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
37602025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37612025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37622025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37632025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37642025-06-04T01:33:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37652025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37662025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
37672025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37682025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37692025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37702025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37712025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
37722025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
37732025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37742025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
37752025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37762025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
37772025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
37782025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
37792025-06-04T01:33:29.066ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1946, tv_nsec: 160741486 } })
37802025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37812025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37822025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37832025-06-04T01:33:29.066ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1946, tv_nsec: 160764557 } })
37842025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
37852025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
37862025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37872025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37882025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37892025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37902025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
37912025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
37922025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
37932025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37942025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37952025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
37962025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.204713ms iteration = 17 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
37972025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37982025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37992025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
38002025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.268236ms iteration = 17 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
38012025-06-04T01:33:29.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38022025-06-04T01:33:29.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38032025-06-04T01:33:29.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38042025-06-04T01:33:29.069ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.785246ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
38052025-06-04T01:33:29.069ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38062025-06-04T01:33:29.069ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:23:29.064954156 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
38072025-06-04T01:33:29.069ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.27598ms iteration = 16 status = {"expiration":"2025-06-04T01:23:29.064954156Z","pruned":[]}
38082025-06-04T01:33:29.070ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38092025-06-04T01:33:29.070ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1946, tv_nsec: 165223551 } })
38102025-06-04T01:33:29.070ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
38112025-06-04T01:33:29.071ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38122025-06-04T01:33:29.071ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38132025-06-04T01:33:29.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38142025-06-04T01:33:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
38152025-06-04T01:33:29.072ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38162025-06-04T01:33:29.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38172025-06-04T01:33:29.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38182025-06-04T01:33:29.074ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38192025-06-04T01:33:29.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38202025-06-04T01:33:29.074ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38212025-06-04T01:33:29.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38222025-06-04T01:33:29.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
38232025-06-04T01:33:29.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38242025-06-04T01:33:29.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38252025-06-04T01:33:29.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38262025-06-04T01:33:29.077ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
38272025-06-04T01:33:29.077ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38282025-06-04T01:33:29.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38292025-06-04T01:33:29.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38302025-06-04T01:33:29.079ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38312025-06-04T01:33:29.080ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1946, tv_nsec: 174744759 } })
38322025-06-04T01:33:29.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38332025-06-04T01:33:29.080ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
38342025-06-04T01:33:29.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
38352025-06-04T01:33:29.081ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38362025-06-04T01:33:29.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38372025-06-04T01:33:29.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38382025-06-04T01:33:29.081ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38392025-06-04T01:33:29.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38402025-06-04T01:33:29.082ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38412025-06-04T01:33:29.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
38422025-06-04T01:33:29.083ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38432025-06-04T01:33:29.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38442025-06-04T01:33:29.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38452025-06-04T01:33:29.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
38462025-06-04T01:33:29.084ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38472025-06-04T01:33:29.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38482025-06-04T01:33:29.084ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38492025-06-04T01:33:29.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38502025-06-04T01:33:29.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38512025-06-04T01:33:29.086ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38522025-06-04T01:33:29.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38532025-06-04T01:33:29.086ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 21.525104ms iteration = 31 status = {"errors":[],"requests_created_ok":[]}
38542025-06-04T01:33:29.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38552025-06-04T01:33:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38562025-06-04T01:33:29.089ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38572025-06-04T01:33:29.089ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38582025-06-04T01:33:29.090ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38592025-06-04T01:33:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38602025-06-04T01:33:29.092ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
38612025-06-04T01:33:29.092ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
38622025-06-04T01:33:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
38632025-06-04T01:33:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
38642025-06-04T01:33:29.092ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38652025-06-04T01:33:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
38662025-06-04T01:33:29.095ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
38672025-06-04T01:33:29.095ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 30.467636ms iteration = 16 status = {"by_rx":{},"error":null}
38682025-06-04T01:33:29.096ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38692025-06-04T01:33:29.097ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 31.629837ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
38702025-06-04T01:33:29.097ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.872472ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38712025-06-04T01:33:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
38722025-06-04T01:33:29.098ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38732025-06-04T01:33:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38742025-06-04T01:33:29.099ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38752025-06-04T01:33:29.099ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
38762025-06-04T01:33:29.099ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
38772025-06-04T01:33:29.099ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 34.223021ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
38782025-06-04T01:33:29.100ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38792025-06-04T01:33:29.101ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38802025-06-04T01:33:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
38812025-06-04T01:33:29.101ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38822025-06-04T01:33:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38832025-06-04T01:33:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38842025-06-04T01:33:29.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38852025-06-04T01:33:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38862025-06-04T01:33:29.103ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38872025-06-04T01:33:29.103ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38882025-06-04T01:33:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38892025-06-04T01:33:29.105ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38902025-06-04T01:33:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38912025-06-04T01:33:29.106ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38922025-06-04T01:33:29.106ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1946, tv_nsec: 201426538 } })
38932025-06-04T01:33:29.107ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
38942025-06-04T01:33:29.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38952025-06-04T01:33:29.107ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38962025-06-04T01:33:29.108ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38972025-06-04T01:33:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38982025-06-04T01:33:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38992025-06-04T01:33:29.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39002025-06-04T01:33:29.113ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
39012025-06-04T01:33:29.113ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
39022025-06-04T01:33:29.113ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
39032025-06-04T01:33:29.113ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39042025-06-04T01:33:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39052025-06-04T01:33:29.115ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
39062025-06-04T01:33:29.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
39072025-06-04T01:33:29.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
39082025-06-04T01:33:29.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
39092025-06-04T01:33:29.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
39102025-06-04T01:33:29.116ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39112025-06-04T01:33:29.116ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
39122025-06-04T01:33:29.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39132025-06-04T01:33:29.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39142025-06-04T01:33:29.117ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 52.252486ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
39152025-06-04T01:33:29.117ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39162025-06-04T01:33:29.119ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39172025-06-04T01:33:29.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39182025-06-04T01:33:29.119ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
39192025-06-04T01:33:29.119ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
39202025-06-04T01:33:29.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
39212025-06-04T01:33:29.128ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
39222025-06-04T01:33:29.128ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.487272ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
39232025-06-04T01:33:29.132ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39242025-06-04T01:33:29.132ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39252025-06-04T01:33:29.132ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39262025-06-04T01:33:29.134ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 68.75788ms iteration = 31 status = {"errors":[],"finish_invoked_ok":[]}
39272025-06-04T01:33:29.135ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 69.581173ms iteration = 16 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
39282025-06-04T01:33:29.136ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39292025-06-04T01:33:29.136ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39302025-06-04T01:33:29.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39312025-06-04T01:33:29.136ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
39322025-06-04T01:33:29.136ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39332025-06-04T01:33:29.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39342025-06-04T01:33:29.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39352025-06-04T01:33:29.138ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39362025-06-04T01:33:29.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39372025-06-04T01:33:29.142ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39382025-06-04T01:33:29.142ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
39392025-06-04T01:33:29.142ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39402025-06-04T01:33:29.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39412025-06-04T01:33:29.145ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 80.010146ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
39422025-06-04T01:33:29.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
39432025-06-04T01:33:29.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
39442025-06-04T01:33:29.147ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39452025-06-04T01:33:29.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39462025-06-04T01:33:29.149ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39472025-06-04T01:33:29.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39482025-06-04T01:33:29.154ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39492025-06-04T01:33:29.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39502025-06-04T01:33:29.157ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39512025-06-04T01:33:29.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39522025-06-04T01:33:29.157ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39532025-06-04T01:33:29.158ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 93.754941ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
39542025-06-04T01:33:29.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39552025-06-04T01:33:29.159ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39562025-06-04T01:33:29.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39572025-06-04T01:33:29.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39582025-06-04T01:33:29.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39592025-06-04T01:33:29.168ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
39602025-06-04T01:33:29.168ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
39612025-06-04T01:33:29.168ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 103.118285ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
39622025-06-04T01:33:29.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39632025-06-04T01:33:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39642025-06-04T01:33:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
39652025-06-04T01:33:29.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39662025-06-04T01:33:29.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39672025-06-04T01:33:29.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
39682025-06-04T01:33:29.177ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39692025-06-04T01:33:29.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39702025-06-04T01:33:29.178ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39712025-06-04T01:33:29.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39722025-06-04T01:33:29.181ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
39732025-06-04T01:33:29.181ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
39742025-06-04T01:33:29.181ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39752025-06-04T01:33:29.181ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 116.524991ms iteration = 34 status = {"physical_disks_added":0}
39762025-06-04T01:33:29.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39772025-06-04T01:33:29.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39782025-06-04T01:33:29.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39792025-06-04T01:33:29.206ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
39802025-06-04T01:33:29.206ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
39812025-06-04T01:33:29.206ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
39822025-06-04T01:33:29.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39832025-06-04T01:33:29.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39842025-06-04T01:33:29.212ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
39852025-06-04T01:33:29.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
39862025-06-04T01:33:29.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
39872025-06-04T01:33:29.350ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
39882025-06-04T01:33:29.350ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39892025-06-04T01:33:29.351ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "925af9d2-4aa5-4e74-8996-2d5d62465021", "content-length": "372", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
39902025-06-04T01:33:29.351ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39912025-06-04T01:33:29.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39922025-06-04T01:33:29.358ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
39932025-06-04T01:33:29.366ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
39942025-06-04T01:33:29.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
39952025-06-04T01:33:29.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39962025-06-04T01:33:29.370ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39972025-06-04T01:33:29.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39982025-06-04T01:33:29.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
39992025-06-04T01:33:29.372ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40002025-06-04T01:33:29.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40012025-06-04T01:33:29.374ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 309.701503ms iteration = 7 status = {"nsuccess":5}
40022025-06-04T01:33:29.375ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40032025-06-04T01:33:29.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40042025-06-04T01:33:29.376ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40052025-06-04T01:33:29.378ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
40062025-06-04T01:33:29.378ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40072025-06-04T01:33:29.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40082025-06-04T01:33:29.382ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40092025-06-04T01:33:29.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
40102025-06-04T01:33:29.382ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
40112025-06-04T01:33:29.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40122025-06-04T01:33:29.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
40132025-06-04T01:33:29.386ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
40142025-06-04T01:33:29.386ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
40152025-06-04T01:33:29.386ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
40162025-06-04T01:33:29.386ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 321.507604ms iteration = 17 status = {"generation":1}
40172025-06-04T01:33:29.393ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
40182025-06-04T01:33:29.394ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
40192025-06-04T01:33:29.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
40202025-06-04T01:33:29.394ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40212025-06-04T01:33:29.394ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "845e30fa-6c08-47d7-850b-6b9321da33c4", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
40222025-06-04T01:33:29.394ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40232025-06-04T01:33:29.395ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40242025-06-04T01:33:29.396ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40252025-06-04T01:33:29.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
40262025-06-04T01:33:29.399ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
40272025-06-04T01:33:29.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
40282025-06-04T01:33:29.401ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
40292025-06-04T01:33:29.401ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 336.790683ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
40302025-06-04T01:33:29.402ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
40312025-06-04T01:33:29.404ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "a72e29aa-cafe-4f7c-9b35-4e6c0e2e99c7", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
40322025-06-04T01:33:29.404ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
40332025-06-04T01:33:29.404ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "c96d25f0-1c80-4f4d-8d69-586156931fe8", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
40342025-06-04T01:33:29.404ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
40352025-06-04T01:33:29.406ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 341.178427ms iteration = 4 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:23:29.617912377Z"},{"operation":"list","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:23:29.618137944Z"},{"generation":1,"operation":"put_config","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:28:29.568314050Z"},{"operation":"list","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:28:29.568532476Z"},{"generation":1,"operation":"put_config","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:33:29.404333662Z"},{"operation":"list","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:33:29.404696202Z"}]}
40362025-06-04T01:33:29.418ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
40372025-06-04T01:33:29.418ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40382025-06-04T01:33:29.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40392025-06-04T01:33:29.418ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "531210cb-fd7c-473d-998d-3482aaf8a096", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
40402025-06-04T01:33:29.419ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40412025-06-04T01:33:29.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40422025-06-04T01:33:29.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40432025-06-04T01:33:29.427ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
40442025-06-04T01:33:29.428ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40452025-06-04T01:33:29.428ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
40462025-06-04T01:33:29.428ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 363.316208ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
40472025-06-04T01:33:29.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
40482025-06-04T01:33:29.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40492025-06-04T01:33:29.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40502025-06-04T01:33:29.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40512025-06-04T01:33:29.434ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
40522025-06-04T01:33:29.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40532025-06-04T01:33:29.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40542025-06-04T01:33:29.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40552025-06-04T01:33:29.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40562025-06-04T01:33:29.441ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40572025-06-04T01:33:29.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40582025-06-04T01:33:29.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40592025-06-04T01:33:29.443ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40602025-06-04T01:33:29.444ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40612025-06-04T01:33:29.447ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
40622025-06-04T01:33:29.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40632025-06-04T01:33:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40642025-06-04T01:33:29.447ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
40652025-06-04T01:33:29.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40662025-06-04T01:33:29.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40672025-06-04T01:33:29.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40682025-06-04T01:33:29.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 385.12264ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
40692025-06-04T01:33:29.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
40702025-06-04T01:33:29.450ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40712025-06-04T01:33:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40722025-06-04T01:33:29.453ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40732025-06-04T01:33:29.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
40742025-06-04T01:33:29.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40752025-06-04T01:33:29.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40762025-06-04T01:33:29.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40772025-06-04T01:33:29.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40782025-06-04T01:33:29.458ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40792025-06-04T01:33:29.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40802025-06-04T01:33:29.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40812025-06-04T01:33:29.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40822025-06-04T01:33:29.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40832025-06-04T01:33:29.460ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40842025-06-04T01:33:29.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40852025-06-04T01:33:29.461ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
40862025-06-04T01:33:29.461ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 396.656904ms iteration = 7 status = {"blueprint_id":"380e7fae-5d5a-479a-b890-8108906f1718","inventory_collection_id":"38598be5-1af3-47c9-8fd1-35a15c522a32","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
40872025-06-04T01:33:29.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
40882025-06-04T01:33:29.464ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40892025-06-04T01:33:29.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40902025-06-04T01:33:29.465ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
40912025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 400.044628ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
40922025-06-04T01:33:29.465ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
40932025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
40942025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8e0f27fd-db00-4da2-b89c-1082351140cb", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
40952025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40962025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
40972025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40982025-06-04T01:33:29.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40992025-06-04T01:33:29.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41002025-06-04T01:33:29.466ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
41012025-06-04T01:33:29.466ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41022025-06-04T01:33:29.467ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "29f5483f-d638-4204-9a6f-7cea0f50b16b", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
41032025-06-04T01:33:29.467ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41042025-06-04T01:33:29.467ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 402.037785ms iteration = 31 status = 0
41052025-06-04T01:33:29.468ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41062025-06-04T01:33:29.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41072025-06-04T01:33:29.469ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41082025-06-04T01:33:29.469ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41092025-06-04T01:33:29.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41102025-06-04T01:33:29.471ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41112025-06-04T01:33:29.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41122025-06-04T01:33:29.475ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41132025-06-04T01:33:29.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41142025-06-04T01:33:29.481ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
41152025-06-04T01:33:29.481ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41162025-06-04T01:33:29.482ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41172025-06-04T01:33:29.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41182025-06-04T01:33:29.484ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9edb27de-bf18-4f00-86d5-7275328ce567", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
41192025-06-04T01:33:29.485ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41202025-06-04T01:33:29.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41212025-06-04T01:33:29.487ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 422.857209ms iteration = 35 status = {}
41222025-06-04T01:33:29.487ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41232025-06-04T01:33:29.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41242025-06-04T01:33:29.490ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41252025-06-04T01:33:29.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41262025-06-04T01:33:29.492ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41272025-06-04T01:33:29.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
41282025-06-04T01:33:29.497ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41292025-06-04T01:33:29.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41302025-06-04T01:33:29.499ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41312025-06-04T01:33:29.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41322025-06-04T01:33:29.501ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41332025-06-04T01:33:29.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41342025-06-04T01:33:29.504ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41352025-06-04T01:33:29.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41362025-06-04T01:33:29.510ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41372025-06-04T01:33:29.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41382025-06-04T01:33:29.513ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41392025-06-04T01:33:29.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41402025-06-04T01:33:29.515ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41412025-06-04T01:33:29.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41422025-06-04T01:33:29.518ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41432025-06-04T01:33:29.524ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
41442025-06-04T01:33:29.524ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41452025-06-04T01:33:29.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41462025-06-04T01:33:29.525ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
41472025-06-04T01:33:29.525ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
41482025-06-04T01:33:29.525ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41492025-06-04T01:33:29.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41502025-06-04T01:33:29.535ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) }
41512025-06-04T01:33:29.535ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 470.741774ms iteration = 17 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
41522025-06-04T01:33:29.537ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41532025-06-04T01:33:29.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41542025-06-04T01:33:29.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41552025-06-04T01:33:29.538ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41562025-06-04T01:33:29.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41572025-06-04T01:33:29.539ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 474.511519ms iteration = 31 status = {}
41582025-06-04T01:33:29.540ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41592025-06-04T01:33:29.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41602025-06-04T01:33:29.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41612025-06-04T01:33:29.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41622025-06-04T01:33:29.544ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
41632025-06-04T01:33:29.544ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
41642025-06-04T01:33:29.544ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
41652025-06-04T01:33:29.544ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 478.696657ms iteration = 17 status = {"generation":2}
41662025-06-04T01:33:29.545ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
41672025-06-04T01:33:29.545ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41682025-06-04T01:33:29.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41692025-06-04T01:33:29.548ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
41702025-06-04T01:33:29.548ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1630d651-b435-4a74-b460-453a2a3db77a", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41712025-06-04T01:33:29.548ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41722025-06-04T01:33:29.549ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41732025-06-04T01:33:29.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41742025-06-04T01:33:29.550ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "424d65c4-6b9a-4567-a8e3-6bf9605c9f6a", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41752025-06-04T01:33:29.550ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41762025-06-04T01:33:29.551ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41772025-06-04T01:33:29.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41782025-06-04T01:33:29.553ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "8c6a65aa-2e7e-45f2-b995-da51f52509c5", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41792025-06-04T01:33:29.553ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41802025-06-04T01:33:29.554ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41812025-06-04T01:33:29.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41822025-06-04T01:33:29.555ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41832025-06-04T01:33:29.555ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 490.515918ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
41842025-06-04T01:33:29.555ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41852025-06-04T01:33:29.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41862025-06-04T01:33:29.556ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41872025-06-04T01:33:29.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41882025-06-04T01:33:29.557ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
41892025-06-04T01:33:29.557ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41902025-06-04T01:33:29.557ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8c4621ec-7b24-4738-876a-6a64e90611c9", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41912025-06-04T01:33:29.557ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41922025-06-04T01:33:29.558ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41932025-06-04T01:33:29.559ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41942025-06-04T01:33:29.559ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41952025-06-04T01:33:29.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41962025-06-04T01:33:29.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41972025-06-04T01:33:29.560ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e580a354-34b7-47a9-90c9-9dc31dc318d4", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41982025-06-04T01:33:29.560ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41992025-06-04T01:33:29.560ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42002025-06-04T01:33:29.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42012025-06-04T01:33:29.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42022025-06-04T01:33:29.562ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42032025-06-04T01:33:29.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42042025-06-04T01:33:29.562ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
42052025-06-04T01:33:29.562ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 497.549476ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
42062025-06-04T01:33:29.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42072025-06-04T01:33:29.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42082025-06-04T01:33:29.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42092025-06-04T01:33:29.564ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42102025-06-04T01:33:29.567ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
42112025-06-04T01:33:29.567ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42122025-06-04T01:33:29.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42132025-06-04T01:33:29.569ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42142025-06-04T01:33:29.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42152025-06-04T01:33:29.570ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42162025-06-04T01:33:29.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42172025-06-04T01:33:29.572ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42182025-06-04T01:33:29.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
42192025-06-04T01:33:29.576ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42202025-06-04T01:33:29.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42212025-06-04T01:33:29.583ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
42222025-06-04T01:33:29.583ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 517.822615ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
42232025-06-04T01:33:29.586ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42242025-06-04T01:33:29.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
42252025-06-04T01:33:29.705ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
42262025-06-04T01:33:29.708ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c82c919-f9a3-42e3-b085-d02da6fbdb52", "content-length": "691", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
42272025-06-04T01:33:29.708ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
42282025-06-04T01:33:29.708ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
42292025-06-04T01:33:29.708ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
42302025-06-04T01:33:29.709ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 644.645155ms iteration = 33 status = {}
42312025-06-04T01:33:29.710ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 645.1905ms iteration = 31 status = null
42322025-06-04T01:33:29.723ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 56513233-0589-4f1f-bfd3-458a6b6602bf, kind: Service { id: 2d727093-e092-497f-a1e2-dd76c1241db0 }, name: Name("external-dns-2d727093-e092-497f-a1e2-dd76c1241db0"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 52])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cf91204e-797e-40be-ad63-a6e7a9ec29d5, kind: Service { id: f6bcd2c3-045d-412e-b32f-07c614adbb05 }, name: Name("external-dns-f6bcd2c3-045d-412e-b32f-07c614adbb05"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 176])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 14b441f3-45c6-4d21-ba32-0c8a69c8e315, kind: Service { id: 7716fd65-e841-4c70-8dfc-958c8d9211a5 }, name: Name("nexus-7716fd65-e841-4c70-8dfc-958c8d9211a5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 10])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72192272-d1eb-4f52-9910-b5b8aafb4794, kind: Service { id: c0229563-2086-4bd0-82c1-da8e271b757f }, name: Name("nexus-c0229563-2086-4bd0-82c1-da8e271b757f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 195, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bcf19bf5-acd4-44fd-9779-d38eb96f2b74, kind: Service { id: 34942533-fcec-49f1-bc91-3bae11367eac }, name: Name("nexus-34942533-fcec-49f1-bc91-3bae11367eac"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
42332025-06-04T01:33:29.723ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42342025-06-04T01:33:29.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42352025-06-04T01:33:29.728ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42362025-06-04T01:33:29.731ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
42372025-06-04T01:33:29.731ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42382025-06-04T01:33:29.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42392025-06-04T01:33:29.734ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
42402025-06-04T01:33:29.734ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 56513233-0589-4f1f-bfd3-458a6b6602bf, kind: Service { id: 2d727093-e092-497f-a1e2-dd76c1241db0 }, name: Name("external-dns-2d727093-e092-497f-a1e2-dd76c1241db0"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 52])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cf91204e-797e-40be-ad63-a6e7a9ec29d5, kind: Service { id: f6bcd2c3-045d-412e-b32f-07c614adbb05 }, name: Name("external-dns-f6bcd2c3-045d-412e-b32f-07c614adbb05"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 176])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 14b441f3-45c6-4d21-ba32-0c8a69c8e315, kind: Service { id: 7716fd65-e841-4c70-8dfc-958c8d9211a5 }, name: Name("nexus-7716fd65-e841-4c70-8dfc-958c8d9211a5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 10])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72192272-d1eb-4f52-9910-b5b8aafb4794, kind: Service { id: c0229563-2086-4bd0-82c1-da8e271b757f }, name: Name("nexus-c0229563-2086-4bd0-82c1-da8e271b757f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 195, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bcf19bf5-acd4-44fd-9779-d38eb96f2b74, kind: Service { id: 34942533-fcec-49f1-bc91-3bae11367eac }, name: Name("nexus-34942533-fcec-49f1-bc91-3bae11367eac"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
42412025-06-04T01:33:29.734ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
42422025-06-04T01:33:29.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42432025-06-04T01:33:29.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42442025-06-04T01:33:29.773ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
42452025-06-04T01:33:29.773ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42462025-06-04T01:33:29.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42472025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
42482025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
42492025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
42502025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
42512025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42522025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42532025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42542025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42552025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42562025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42572025-06-04T01:33:29.787ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
42582025-06-04T01:33:29.787ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42592025-06-04T01:33:29.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42602025-06-04T01:33:29.790ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
42612025-06-04T01:33:29.790ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
42622025-06-04T01:33:29.874ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 4adf0198-0a8a-47c8-a668-dc22afd01e24, time_created: 2025-06-04T01:18:49.382391Z, time_modified: 2025-06-04T01:18:49.382391Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
42632025-06-04T01:33:29.874ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
42642025-06-04T01:33:29.885ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a261b5d8-d095-417e-bc7f-d152bcaa751c", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
42652025-06-04T01:33:29.885ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
42662025-06-04T01:33:29.885ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
42672025-06-04T01:33:29.886ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4cee7ce-46d0-45cd-9b75-7adea6faf454", "content-length": "682", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
42682025-06-04T01:33:29.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42692025-06-04T01:33:29.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42702025-06-04T01:33:29.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42712025-06-04T01:33:29.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42722025-06-04T01:33:29.908ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42732025-06-04T01:33:29.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42742025-06-04T01:33:29.911ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42752025-06-04T01:33:29.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42762025-06-04T01:33:29.912ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42772025-06-04T01:33:29.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42782025-06-04T01:33:29.913ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42792025-06-04T01:33:29.914ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42802025-06-04T01:33:29.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42812025-06-04T01:33:29.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
42822025-06-04T01:33:29.916ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
42832025-06-04T01:33:29.916ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3adcf8b1-d559-451a-b336-728df20b00d7", "date": "Wed, 04 Jun 2025 01:33:29 GMT"} })
42842025-06-04T01:33:29.919ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:495
42852025-06-04T01:33:29.919ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 853.995722ms
42862025-06-04T01:33:29.919ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 854.006032ms iteration = 4 status = {}
42872025-06-04T01:33:29.922ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 856.977766ms iteration = 31 status = {}
42882025-06-04T01:33:31.116ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:47804
42892025-06-04T01:33:31.116ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:47804 req_id = 6d235f52-4aae-4ce8-850d-ad252bc6ccab roles = RoleSet { roles: {} } uri = /metrics/producers
42902025-06-04T01:33:31.116ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:47804 req_id = 6d235f52-4aae-4ce8-850d-ad252bc6ccab resource = Database result = Ok(()) uri = /metrics/producers
42912025-06-04T01:33:31.155ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 0d417b9f-81d7-4f7f-afd4-a2f10841f4be
42922025-06-04T01:33:31.156ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 39768 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:47804 req_id = 6d235f52-4aae-4ce8-850d-ad252bc6ccab response_code = 201 uri = /metrics/producers